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ABSTRACT 



This guide provides advice on effective use of assessments 
in adult basic and literacy education programs. Chapter 1 considers different 
types of standardized tests, their purposes, their advantages and 
limitations, misuses of standardized tests, and other types of assessments. 
Chapter 2 on informal assessments defines alternative and informal assessment 
and discusses types of informal assessment, their advantages and limitations, 
and design of informal assessments. Chapter 3 addresses use of assessments to 
document learner progress for program accountability, including types of 
assessment appropriate for program evaluation and accountability, how to 
standardize a locally developed assessment, and use of qualitative 
information for program accountability. Chapter 4 covers use of assessments 
to screen learners and to guide and evaluate instruction. The focus of 
chapter 5 is assessment of adults for whom English is a second language. 
Chapter 6 covers assessment strategies to use with students with learning 
difficulties and making accommodations. Chapter 7 describes the information 
that should be included in a manual for a commercially developed test and 
provides a checklist to examine a test to decide if it is appropriate for 
one's assessment needs. Chapter 8 discusses factors within the learner and 
the program that affect the testing situation and addresses supportive 
environments. Appendixes contain the following: test summaries for 11 tests, 
annotated list of 9 resource organizations, and disability definitions and 
checklists. Introduction and individual chapters contain references. (YLB) 
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Introduction 



Two national 
studies of ABLE 
programs concluded 
that programs have 
not been effectively 
collecting or 
analyzing learner 
progress data. 



Funders and Researchers Question the Effectiveness of 
ABLE Programs 

Nationally, funders and researchers are asking critical 
questions about the effectiveness of adult basic and literacy 
education (ABLE) programs. Two national studies evaluating 
the performance and quality of ABLE programs have released 
reports in the past few years. Both studies concluded that 
evaluating these programs is difficult, because programs have 
not collected or analyzed information about their activities 
effectively. Specifically, data on learner progress are often 
incomplete, inaccurate, or difficult to interpret. 



Researchers and 
practitioners have 
questioned the 
validity of 
assessments used 
in adult education 
programs. 



Other researchers have questioned the validity of assess- 
ments used in adult education programs. Many practitioners 
agree with the researchers, reporting that standardized tests do 
not adequately reflect learners’ progress nor do they provide 
enough useful information to guide instruction. Concerns such 
as these make administering standardized assessments — 
necessary for program accountability — ^more difficult and the 
results less reliable. As a result, ABLE programs have found it 
difficult to collect credible evidence of their effectiveness and 
impact on adults’ acquisition of basic skills and knowledge. 



The lack of 
convincing evidence 
about programs 
substantially 
increasing learners’ 
skills makes it 
difficult to justify 
providing adult 
education 
services at public 
expense. 



Furthermore, recent large-scale studies have not found 
significant overall impacts of adult education on literacy 
abilities. The lack of convincing evidence about programs 
substantially increasing learners’ literacy skills, coupled with 
some adult educators’ resistance to collecting standardized test 
data, make it difficult for us to justify providing adult educa- 
tion services at public expense. These questions and concerns, 
as well as the expected transition to block grant funding, could 
have a devastating impact on ABLE: funding could be 
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drastically reduced or eliminated, thus leaving adults without 
essential educational services. 



Adult educators 
play a critical part 
in the programs’ 
ability to 
demonstrate 
learner progress. 
They must be skilled 
and knowledgeable 
about effective 
assessment 
practices. 



Why Should We, as Adult Educators, Be Concerned about 
Assessment? 

Most of us believe that we arc doing good work and that 
learners enrolled in our programs arc making — often 
miraculous — ^progress; however, we have not been very 
successful in documenting these successes. Since learner 
progress is one of the indicators of program quality or effec- 
tiveness, we have a responsibility to both the learners in our 
ABLE programs and to ourselves to increase our knowledge 
and skills in the area of assessment. As instructors or tutors in 
ABLE progratrrs, the question of learner progress affects us as 
well as learners. When ABLE programs do not document 
learner progress effectively, it reflects not only on the pro- 
grams’ quality but also on our skills as teachers. We play a 
critical part in oiu programs’ ability to demonstrate learner 
progress. Therefore, we need to expand our skills to document 
learner progress by knowing the various piuposes for assess- 
ment, understanding the types of assessments and how they are 
administered and scored, and using both standardized and 
informal test results to plan and modify instructional plans with 
the learners. 
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Chapter 1: What are the Different 
Types of Standardized Assessments? 
How and Why Should I Use Them? 



Multiple 
approaches to 
assessment are 
necessary to meet 
the iitformation 
needs of all the 
stakeholders. 



The Bureau 
of ABLE 
encourages 
the use of 
standardized 
tests to conduct 
student 
assessments. 



Introduction 

Assessment in adult basic and literacy education (ABLE) 
programs must satisfy multiple stakeholders, each with differ- 
ent purposes for assessment and interests in the outcomes of 
the program. Multiple approaches to assessment are necessary 
to satisfy these various information needs. For example, in 
community adult literacy programs you may think primarily of 
two stakeholders — the learner and the instructor. In the case of 
workplace literacy programs, however, at least two additional 
stakeholders emerge, namely, the union (if one exists) and the 
management of the company. Other stakeholders in any ABLE 
program will be the program administrators and the funding 
agency, such as the state department of education. 

ABLE programs traditionally depend on the results of 
standardized tests to determine changes in participants’ basic 
skills levels and to report pre- and posttest data to the funding 
agency. The Bureau of ABLE encourages the use of standard- 
ized testing to conduct student assessments, reporting that 44% 
of Pennsylvania’s ABLE providers use the Tests of Adult Basic 
Education (TABE) — by far the most commonly used standard- 
ized test in the Ck)mmonwealth. On the other hand, 46% of 
ABLE programs report using “other” assessments, such as 
portfolios or checklists. This high percentage reflects practitio- 
ners’ dissastisfaction with standardized instruments. In recent 
years, practitioners have increasingly relied on observation and 
informal measures as more authentic measures of learner 
progress. 



When selecting assessments, consider these three impor- 
tant questions: What are the purposes of assessment? What 




1.1 



are the assessment information needs of each stakeholder? 
What are the strengths and limitations of the various assess- 
ment instruments for each of these needs? 



What are the Purposes of Assessment? 

Assessment instruments in ABLE programs are used to 
1) gather intake information for screening and placement in an 
instructional program; 2) diagnose each individual’s educa- 
tional strengths and weaknesses for developing an instructional 
plan; and 3) measure learner progress to determine individual 
skill growth as well as to document the gains of the entire 
group for program accountability. 



Intake 
assessments 
yield irformation 
for screening 
and placing 
learners in an 
instructional 
program. 



Intake assessments should be conducted in a welcoming 
and supportive environment. The importance of a successful 
and reassuring first meeting is well-supported in the research as 
a key indicator of whether students will participate in the 
program. Intake assessment first involves an interview to 
determine each individual’s reasons for enrolling, what they 
want to accomplish and learn (their goals), their interests and 
talents, and their educational history. Screening or placement 
testing (such as with the TABE Locator Test, the Adult Place- 
ment Indicator, or a program-developed, informal test) is then 
administered either individually or in a group to provide a 
starting point for instruction. 



Diagnostic testing 
identifies each 
individual’s 
strengths and 
weaknesses for 
instructional 
planning. 



Diagnostic testing is conducted when the student is com- 
fortable in the program, usually within the first month of 
enrollment. This assessment provides more in-depth informa- 
tion about the learner’s specific skill strengths and weaknesses. 
For example, the results of the TABE Complete Battery, the 
Reading Evaluation Adult Diagnosis (READ), and informal 
reading inventories provide such diagnostic information. 
Instructors and tutors then use the information to build on 
students’ strengths and to emphasize skill areas that are weak 
or missing. 
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Assessments to 
measure learner 
progress document 
the effectiveness of 
the program for 
individuals and for 
the entire group. 



Instructors need 
assessment 
irtformation for 
screening and 
placing learners, 
instructional 
planning, and 
documenting 
instructional 
effectiveness. 
This irtformation 
shoidd be shared 
with students on a 
regular basis. 



Funding agents 
need assessment 
data for 
accountability 
purposes. 



O 




Assessments that measure learners’ progress are con- 
ducted periodically to document the effectiveness of the 
program for individuals and for the entire group. These assess- 
ments may be standardized or informal measures, depending 
on the stakeholders’ needs. It’s important, however, that the 
same assessments used for pretesting be used for interim and 
posttesting. For example, if a student was pretested with a 
certain level of the TABE and also with a locally developed 
assessment, these same instruments should also be used for 
measuring gains. 



Who Needs Assessment Information? 

Instructors need to screen students and place them quickly 
and comfortably in the appropriate programs. They also need 
to diagnose learners’ strengths and weaknesses so they can 
develop and periodically revise a learning plan that fits indi- 
vidual needs. Instructors also need to measure student 
progress in order to demonstrate the effectiveness of their 
instruction to program administrators responsible for reporting 
data to funding agents and policymakers. Students, too, are 
very interested in knowing more about their educational 
strengths and weaknesses and their progress in the instructional 
program. Instructors should share assessment results with 
students on a regular basis to engage them in the learning 
process, thus providing an opportunity for collaborative in- 
structional planning. 

Funding agents require assessment data for accountability 
purposes. Funding agents are not interested in an individual’s 
learning strengths and weaknesses but instead in the progress 
of a group of students. This progress may be demonstrated by 
comparing scores on standardized tests before and after a given 
amount of instruction or time period, or by identifying the 
competencies or objectives that students have mastered during 
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Program 
administrators 
must submit data on 
learner progress to 
the funding agent. 



a given period of time. Program administrators, while they 
may be interested in instructional planning for individuals, also 
must collect assessment data for purposes of accountability. 
They must submit data on learner progress to the funding 
agent. These data also will help to formulate and guide the 
program’s improvement plan. 

The assessment instruments you use must meet the needs 
of all the stakeholders. They should be selected to measure 
what you are teaching and should assist you in documenting 
program effectiveness and for making program changes if 
necessary. The following sections provide information on the 
various assessment instruments and how the data are measured, 
analyzed, and interpreted to report the learning that is occurring 
in programs. 



Standardized 
tests have a 
standard set 
of directions 
in administration, 
scoring, and 
interpreting results. 



What are Standardized Tests? 

Standardized tests, as the name suggests, follow a standard 
set of directions in test administration, scoring, and interpreta- 
tion. Learners located in different parts of the country, and in 
different types of programs, should be assessed under the same 
conditions. Instructors in different programs administer the 
test and score and interpret the data in the same way. The 
results can be compared if the standard instructions are fol- 
lowed and learners are all appropriate for the given level of the 
test 



Standardized tests are developed by testing companies 
which allocate many resources to ensure that tests are valid and 
reliable. Validity means that a test measures what it is sup- 
posed to measure; a test is reliable if similar results are 
obtained each time it is administered to the same learner with 
no intervening instruction. (See Chapter 7 for detailed infor- 
mation concerning reliability and validity.) 



Many standardized 
tests yield grade 
equivalents (e.g., 
third grade reading 
level) which are not 
recommended for 
adult learners. 



Many standardized tests yield a score as a grade equiva- 
lent. Use of a grade equivalent is not recommended when 
working with adults. Grade equivalent scores do not truly 
reflect adults* abilities and may project negative connotations. 
For example, an adult with a third grade equivalent reading 
level does not behave as a third grader and may be quite 
competent and skilled in other areas. (The International 
Reading Association recommends that grade equivalents be 
abandoned for all students regardless of age.) 



Scaled scores 
and percentiles 
reflect the 
distribution of 
your students’ 
scores. 



A better choice is to use scaled scores (not raw scores) or 
percentiles which, in general, reflect the distribution of your 
students’ scores. Specifically, scaled scores, such as those 
provided in the TABE, are derived by taking raw scores and 
converting them to a single interval score (such as 0 through 
999) to permit comparison of performance across levels and 
forms. Percentiles show where individuals place in the distri- 
bution of the norming group. 



Scaled scores, such 
as those in the 
TALS, are 
replacing grade 
equivalents as 
better measures of 
adults’ abilities. 



Recently, standardized adult education tests have been 
influenced by the National Adult Literacy Siuvey or NALS, 
which eliminated grade equivalents; instead, student scores are 
arrayed on a proficiency scale of 500 points. The Educational 
Testing Service, which created the NALS, also created a 
commercial version called the ETS Tests of Applied Literacy 
Skills or TALS which uses the same proficiency scale. Simi- 
larly, the Comprehensive Adult Student Assessment System — 
CASAS and Work Keys (American College Testing) use scaled 
scores instead of grade equivalents. 



Standardized tests 
have become more 
contextual. 



Standardized tests also have become more contextual in 
response to the criticism that they measure generic academic 
skills. The TABE now has a workplace literacy assessment for 
specific occupational clusters. Work Keys is oriented to the 
workplace, and CASAS likewise has a test version to assess 
basic skills in a workplace context 



In swnmary, 
standardized tests 
have standard 
directions for 
administration, 
scoring, and 
interpretation. 
They are valid and 
reliable. 



In summary, standardized tests must have standard instruc- 
tions for administering, scoring, and interpreting data. All 
students should have the same amount of time to take the test, 
test administrators should score all tests using the same criteria, 
and test administrators should come to the same conclusions 
when interpreting the test results. Standardized tests must be 
valid and reliable. In other words, they must measure what 
they are designed to measure. They also must produce similar 
results if an individual takes the test again without having any 
intervening instruction. 



An advantage 
of standardized 
tests is that the 
results can be 
summarized to 
compare program 
effectiveness. 



Advantages and Limitations of StandartUzed Tests 
One advantage of standardized tests is that they yield 
comparable results regardless of location or program type. 
Therefore, funding agents and policymakers often rely on them 
for program accountability. Another advantage is that gains on 
standardized tests may be useful to program administrators and 
instructors to get an objective measure of students’ progress 
compared to an external norming group. Testing companies 
adminis ter their tests to an appropriate target audience that is 
large enough to establish a norming group (which ideally 
should represent your student population). Another advantage 
of standardized tests is that they may show the groups’ progress 
toward meeting specified criteria or competencies. 



A limitation of 
standardized tests 
is that the content 
may be irrelevant 
to what is being 
taught in the 
program. 



A limitation of standardized tests is that the content may 
be irrelevant to what is being taught in the curriculum and 
classroom. For example, a workplace literacy program that 
uses the context of the job for instruction in basic skills should 
be careful in using standardized tests since the curriculum 
probably does not match the content of the tests. The standard- 
ized test may not reflect the actual gains that students are 
making in the program because the test may not be assessing 
the skills that are being taught. (On the other hand, the test 
may assess the same skills that are taught in the instructional 
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Another limitation 
is that the norming 
group for the test 
many not be 
comparable to your 
student population. 



program but in a generic context; then the test may indicate the 
ability of the students to transfer what is learned to literacy 
tasks beyond the workplace.) Another limitation to standard- 
ized tests is that the group of students may be not comparable 
to the norming group. For example, a test may be normed on 
students in general adult basic education programs; however, 
your students may be non-native speakers of English, disabled, 
or institutionalized. Comparisons to the norming group should 
be made only cautiously. 



ABLE 

practitioners may 
incorrectly 
administer a 
standardized test 
in a variety 
of ways. 



Misuses of Standardized Tests 

Sometimes ABLE instructors misuse standardized and 

norm-referenced tests by: 

• ignoring the standardized instructions, giving students as 
much time as they want to complete the test, for example; 
test conditions, then, are no longer standard and the results 
are not comparable to others who are taking the same test. 

• using different tests to measure gains before and after 
instruction; the same test, but a different form, should be 
used for pre- and posttesting since all tests have different 
norming groups and standards. 

• administering standardized tests on fixed dates (say, fall 
and spring) regardless of the amounts of time individuals 
have been in the program; therefore, individuals who have 
been in a program for 15 hours may be compared to those 
who have had 30 hours of instruction. 

• administering standardized pretests before students feel 
comfortable in the program; students tend to test below 
their actual capabilities on the pretest and higher on the 
posttest when they are familiar with the instructional 
setting. The result is that the gains due to instruction are 
not known. 



ABLE 

practitioners may 
misinterpret or 
incorrectly report 
students’ scores on 
standardized tests. 



using an inappropriate level of a standardized test for pre- 
and posttesting. If students take a test that is too difficult, 
they score at the “floor” of the test and cannot show what 
they do know; conversely, if the test is too easy, they score 
at the “ceiling” and cannot show any growth over time. 

reporting only grade equivalent scores which are actually 
only gross estimates of abilities; scaled scores are more 
accurate since they account for the inevitable amount of 
test error in scores. 

comparing students’ test scores to an inappropriate 
norming group; the norming group used by the test 
developer can be identified by reading the test’s technical 
manual. 

averaging median or percentile scores for presenting a 
group’s performance; the only scores that should be 
reported as average or mean scores are scaled and raw 
scores. (Raw scores are limited because they do not reflect 
the normative data and cannot be used across test levels.) 



Norm-referenced 
tests compare your 
learners’ scores to 
those of the 
norming group. 



What are the Various Types of Assessments? 

Traditionally, ABLE programs have used standardized, 
norm-referenced achievement tests in which students’ test 
scores are compared to those of the norming group. Alterna- 
tives to norm-referenced achievement tests include criterion- 
referenced, curriculum-based, performance-based, and other 
informal tests. Some of these assessments are standardized, 
such as the CASAS; however, many are informal tests devel- 
oped by curriculum developers or locally by programs. 



Criterion-referenced assessments have become popular in 
measuring skill mastery as opposed to norm-referenced tests 



Criterion-referenced 
tests indicate 
whether or not 
a student has 
mastered a skill 
at a specific 
level of 
competence 
(say, 80 %). 



(e.g., the TABE). Instead of comparing an individual’s perfor- 
mance to that of others through grade equivalents or scaled 
scores (as in standardized, norm-referenced tests), the 
individual’s performance is compared to his or her earlier 
performance in achieving or mastering a particular skill. In- 
structors find that criterion-referenced testing is useful for 
competency-based instructional planning since these assess- 
ments indicate whether or not an individual has mastered a 
particular skill at a specified level of competence (say, 80 %). 
CASAS is peihaps the best known criterion-referenced literacy 
assessment in adult education; it is also standardized although 
not norm-referenced (since the individual’s performance is not 
compared to that of a group). 



Curriculum-based 
tests measure 
whether or not 
students have 
learned what has 
been taught in a 
given set of 
instructional 
materials. 



Curriculum-based assessment refers to tests that assess 
learning from a given set of curriculum materials. A test at the 
end of a chapter of instructional materials is a type of curricu- 
lum-based assessment. In contrast to criterion-referenced 
assessment, curriculum-based assessment does not measure 
skill mastery. Instead, it measures whether or not the learners 
have learned what has been taught in a given piece or set of 
instructional materials. Like criterion-referenced assessment, 
it may be standardized or informal depending on its develop- 
ment. 



” Performance-based 
assessments 
require students 
to demonstrate 
their knowledge 
or skills. 



Performance-based assessment also is not new, but it is 
enjoying renewed interest as a technique for measuring skill 
applications since it requires demonstration of competency. 
Some performance assessments require paper-and-pencil 
responses; others use computer simulations or real-life demon- 
strations of competency. As a standardized performance-based 
assessment, the National External Diploma Program (Ameri- 
can Council on Education) has been assessing adults’ compe- 
tence acquired through life experiences and self-directed 
learning for many years as an alternative to the GED. New 



approaches to assessing basic skills through computer-based 
simulations of real problems encountered in a workplace have 
been recently devised, such as A Day in the Life. .Assessment 
(Curriculum Associates). Computer-based assessment also 
standardizes test administration and scoring. 



Informal 

assessments, such 
as portfolios, are 
more relevant to 
the assessment 
needs of learners. 



Irtformal assessments, such as portfolios, have become 
popular as a supplement — and sometimes a replacement — to 
standardized tests. Alternative assessments that focus on 
learners’ progress toward either program-specific indicators or 
self-selected goals are seen by some researchers and practitio- 
ners as an effective method for assessing learners in authentic 
or meaningful contexts. Often, teachers develop informal 
assessment tools, such as interviews, checklists, or portfolios, 
that are useful with particular individuals or in a given curricu- 
lum. While they may be relevant to the assessment needs of 
the individual and program, and therefore are valid, they lack 
demonstrated reliability. Nevertheless, numerous adult educa- 
tors have argued for this change of direction in assessment. 



The usual 
standardized, 
norm-referenced 
tests of adult basic 
skills, such as 
the TABE, may 
not be expropriate 
for disced or 
ESL learners. 



Finally, special attention should be given to assessments 
for learners for whom English is a Second Language (ESL) or 
who may be disabled. The usual standardized norm-referenced 
tests of adult basic skills may not be appropriate for their needs 
(see Chapters 5 and 6). Dennis (1995) found that ESL provid- 
ers in Pennsylvania indicated a general satistfaction with 
commercially available ESL tests, but a relatively heavy use of 
“assessment tools developed in-house.” Also, Cooper (1992) 
has developed an alternative assessment instrument specifically „ 
for learning disabled adults and provided staff development 
sessions regarding its use. On the other hand, little has been 
done to address assessment issues concerning other adults with 
disabilities (e.g., mental retardation, mental illnesses, or physi- 
cal disabilities). 



Summary 

ABLE programs must select a variety of standardized 
and informal tests based on their assessment purposes, the 
information needs of all stakeholders, and the strengths and 
limitations of the various tests. Assessment practices should 
effectively measure what you are teaching and should assist 
you in documenting program effectiveness and for making 
program changes if necessary. For example, intake and screen- 
ing assessments (such as the TABE Locator Test or a program- 
developed informal test) provide a starting point for instruction. 
Diagnostic tests (such as the TABE Complete Battery or an 
informal reading inventory) guide ongoing instructional plan- 
ning. Program administrators and funding agents use standard- 
ized test data (such as pre- and posttest data on the TABE or 
CASAS ) to document and compare program effectiveness. 
Finally, all ABLE program staff and administrators should 
consistently and carefully document their assessment practices. 



References 

Cooper, R. (1992). Staff development course about learning 
problems in adults (Final Report). Harrisburg, PA: Pennsyl- 
vania Department of Education. Bureau of ABLE. 

Dennis, R. (1996). An examination of English as a second 
language assessment tools utilized by adult literacy provid- 
ers in Pennsylvania. PAACE Journal of Lifelong Learning, 

5, 1-13. 

Salvia, J., & Ysseldyke, J. E. (1995). Assessment (Sixth 
Edition). Boston, MA: Houghton Mifflin Co. 

Salvia^ J., & Hughes, C. (1990). Curriculum-based assess- 
ment: Testing what is taught. New York, NY: MacMillan 
Publishing Co. 



Sticht,T. G. (1990). Testing and assessment in adult basic 
education and English as a second language programs. San 
Diego, CA: Applied Behavioral & Cognitive Sciences, Inc. 

Venezky, R. L., Bristow, P. S., & Sabatini, J. P. (1994). Mea- 
suring gain in adult literacy programs. Philadelphia, PA: 
National Ctenter on Adult Literacy, University of Pennsylva- 
nia. 

Zellers, R. (1986). A resource guide of tests for adult basic 
education teachers. Harrisburg, PA: Pennsylvania Depart- 
ment of Education, Bureau of Adult Basic and Literacy 
Education. 



Chapter 2 



What are the Different Types 
of Informal Assessments? 
How Should I Use Them? 




22 



Chapter 2: What are the Different 
Types of Informal Assessments? 
How Should I Use Them? 



Adult education 
instructors and 
learners are 
interested in 
assessment 
itrformation that 
shows individual 
skill progress. 



Itrformal 
assessments, such 
as portfolios, 
checklists, and 
self-assessments, 
assist in collecting 
information about 
an individual’s 
progress, and can 
also guide 
instruction and help 
learners set goals. 



Introduction 

Program administrators and funders regularly require 
information about learner progress, usually in the form of data 
collected through the administration of standardized instru- 
ments which will provide generalizable information. These 
stakeholders are most interested in the overall effectiveness of 
ABLE programs. On the other hand, adult education instruc- 
tors and learners are most interested in individual progress. As 
a result, they may reject standardized instruments as inad- 
equate, because the tests do not provide sufficient detail about 
the learners’ skills, making it difficult to plan instruction. 
Instructors also may believe that the standardized test being 
used does not adequately document learner gains. It may seem 
that instmctors and learners are at cross purposes with adminis- 
trators and funders. In fact, these differences acmally reflect 
the various purposes for assessment. 

Efforts to supplement standardized test data often focus on 
the use of informal assessments. For example, programs might 
adopt portfolio assessment, teacher-developed checklists and 
self-assessments, curriculum-based assessments, program- 
designed interviews, and commercially available rating sheets 
to assist in collecting additional information about learners’ 
progress. Informal instruments, often developed by program 
staff, can involve instructors and learners in the assessment 
process. They can help learners set goals and guide instruction. 
They also can document changes in a learner’s self-esteem or 
track small steps toward larger goals. 



Unfortunately, these same assessment activities fall short in 
meeting requirements for program accountability. Informal 
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Irtformal 
assessments are 
valuable tools but 
not replacements 
for standardized 
tests. 



instruments are valid within a program; however, the resulting 
data do not translate across programs or often even across 
learners in the same program. In many instances, little evi- 
dence exists that the instruments are reliable or even that 
program staff agree on the purpose or outcomes expected from 
the assessment. As a result, informal assessments are valuable 
tools but not replacements for standardized tests. 



What are Informal Assessments? 

The terms informal assessment and alternative assessment 
are often used interchangeably. In fact, while all informal 
assessments are alternative assessments, not all alternative 
assessments are informal. 



Alternative 
assessments 
are often 
process-oriented, 
open-ended, 
and ongoing. 



Defining Al ternative Assessment 

Alternative assessment refers to two trends in testing. 

First, alternative assessments refer to tests that are not tradi- 
tional norm-referenced achievement instruments. Alternative 
assessments, in this case, can be standardized or informal and 
include criterion-referenced, curriculum-based, performance- 
based instraments, and various forms of portfolios. Secondly, 
alternative assessment refers to assessment practices that vary 
from traditional approaches often associated with the use of 
standardized tests. For example, Auerbach (1992) explains 
that alternative assessments are most often process-oriented, 
open-ended, and ongoing. They also involve learners in the 
assessment process and are conducted in a supportive environ- 
ment. Herman, Aschbacher, and Winters (1992) add that 
alternative assessments often involve higher-level thinking and 
problem-solving skills as well as real-world applications. 
Standardized tests rarely include these attributes; however, 
ABLE providers can adopt more learner-involved and support- 
ive approaches to assessment — even when using traditional 
standardized tests (See Chapter 8 for additional information). 



O 
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Marty informal 
tests are teacher- 
constructed. They 
are designed to 
reflect the skills 
and knowledge 
presented in the 
curriculum, the 
instructional or 
educational goals 
of the program, 
or learners’ self- 
selected goals. 



Informal Assessment 

Informal tests can be commercial (e.g., curriculum-based 
tests included with instructional materials) but many are 
teacher-constructed. Instructors — and, on occasion, learners 
working with instructors — design informal tests that reflect the 
skills and knowledge presented in the curriculum, the general 
instructional/educational goals of the program, or learners’ self- 
selected goals. For example, an instructor may design an infor- 
mal performance-based assessment to determine learners’ 
ability to perform activities using basic skills and knowledge 
taught in a customized workplace literacy program (e.g.. Learn- 
ers read tables and charts to solve a specific workplace prob- 
lem). In another setting, program staff and parents may design 
an informal criterion-referenced checklist to determine progress 
in adopting selected family literacy practices (e.g.. Parents self- 
report or instructors observe changes in parent-child reading 
practices or active participation in children’s schooling.). 
Finally, learners may select personal goals, and with the in- 
structor, develop milestones that document progress toward the 
goals or criteria for determining achievement of the goals (e.g.. 
The learner’s goal may be to learn personal money manage- 
ment. Milestones might include learning to perform math 
operations associated with budgets and developing a family 
budget.). 



Irrformal 

assessments provide 
an opportunity to 
integrate ongoing 
assessment with 
instruction. 



What are the Different Types of Informal Assessment? 

Tests usually measure a learner’s performance at one point 
in time. Informal assessment instruments provide an opportu- 
nity to integrate ongoing assessment with the instructional 
process. Informal assessments can help track progress as well 
as finished products as learners move toward improved skills. 

The two practices described below encourage instructors 
and learners to think about and record their observations, using 
the information to discuss educational progress. 
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Observations may 
be recorded in 
unstructured or 
semi-structured 
anecdotal records 
or journals. 
Frequently discuss 
observations with 
learners, asking for 
their reactions and 
planning further 
instructional 
activities. 



Observation 

Instructors purposefully observe learners as they work, 
looking for effective use of skills or areas that require addi- 
tional work. Observations may be recorded in unstructured 
and open-ended or semi- structured anecdotal records or 
Journals. Unstructured records consist of any notes that the 
instructor wishes to include about the learner’s use of basic 
skills For example, comments may vary depending on the 
literacy task or on the instructor’s knowledge of previous 
assessment results. Semi-structured records might use an 
observation guide consisting of specific literacy skills to 
observe and a simple rating of plus (+) or minus (-) to indicate 
whether the skill was observed or not. You can find samples 
of observation guides in published assessment guides. After 
you make a series of observations, meet with learners to 
discuss your observations, asking learners for their reactions 
and planning further instructional activities. 



Self-assessments 
help learners 
ejqylore how their 
work shows 
progress or the 
need for additional 
practice. 



Self-Assessments 

You should encourage adults to reflect on their learning 
and explore how and why their work shows progress or the 
need for additional practice. The process of reflection engages 
learners in the educational process, demanding that they 
evaluate their own work, think of possible solutions to identi- 
fied problem areas, and become responsible for progress 
toward and eventual achievement of personal goals. 



Self-assessments 
can include 
reflective Journals, 
checklists, and 
open-ended 
surveys. 



Self-assessments can take various forms, including, for 
e xamp le, unstructured interviews, reflective journals, or check- 
lists and open-ended surveys. Holt (1994) includes several 
examples of open-ended surveys and questionnaires; these 
instruments focus on English as a Second Language learners 
and family literacy programs but are easily adaptable to other 
settings. 
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Information gathered through the following informal 
assessments also can be used to plan instruction or to track and 
document progress toward specified criteria or self-selected 
goals. 



Irtformal reading 
inventories can be 
used to diagnose a 
learner’s skills in 
oral and silent 
reading, especially 
to analyze miscues 
to guide instruction. 



Informal Reading Inventories 

Informal reading inventories (IRIs) usually consist of a 
word list and a series of graded reading passages. The learner 
reads the word list orally. Depending on the inventory’s 
guidelines, the reading is discontinued when the learner 
mispronounces a specified number of words. This indicates 
where the learner should begin reading the passages, usually 
one paragraph to be read orally and one to be read silently at 
each grade level. You record miscues (e.g., noting omitted or 
inserted words and deleted word endings) observed during oral 
reading. For both the oral and silent reading passages, you ask 
several vocabulary and comprehension questions, record the 
responses, and stop the assessment when the learner misses a 
specified number of questions. IRIs can be used to diagnose a 
learner’s skill in oral and silent reading comprehension as well 
as identifying reading miscues. This information can then be 
used to develop an individual education plan and identify 
appropriate instructional strategies. 



Retell exercises help 
instructors discern 
learners' text 
comprehension 
and facility with 
language. 



Retell Exercises 

Retell exercises require your asking learners to tell you 
what they have read in a selected passage. Although analyz- 
ing responses can be difficult, retelling can provide informa- 
tion about learners’ text comprehension, use of metacognitive 
strategies (e.g., connections with prior knowledge, generaliza- 
tions to real world situations), and facility with language (e.g., 
organization, audience). Herrmann (1994) includes a format 
for recording learner responses during retelling. 



27 

o 

ERIC 



2.5 



Analysis of writing 
samples provides 
information on 
specific writing 
skills and can be 
used to giude 
instruction. 



Writing Samples 

You can collect samples over time to evaluate learners* 
growth as writers. Analysis of the samples can provide infor- 
mation on specific skills (e.g., use of grammar, spelling, sen- 
tence and paragraph structure, use of descriptive words) and 
direction for instruction and guided practice. Checklists and 
criteria are available to evaluate various aspects of adults’ 
writing. For example. Bear (1987), and MacKillop and 
Holzman (1990) include criteria for evaluating writing samples 
as well as several checklists. 



Curriculum-based 
assessments 
are closely linked 
to the curriculum. 
However, don't 
overlook other 
assessments that 
provide more 
useful irformation 
about learners' 
skills and progress. 



Curriculum- Based Assessments 

You can use curriculum-based assessments included with 
some commercial instructional materials, or you can design 
assessments based on curricula developed in your program. 
Although curriculum-based assessments are closely linked to 
the ciuriculum, they have drawbacks. Curriculum-based 
assessments, especially in materials written for adults at the 
begitming levels of literacy, often rely on literal comprehension 
and simple recall. The ability to recall and understand infor- 
mation expressed literally in the text is an important skill; 
however, it provides only limited information about learners’ 
reading comprehension. Also, instructors who depend on 
curriculum-based assessments may overlook other informal 
and standardized assessments that might provide more useful 
information about learners’ skills and progress. 



When developing 
curriculum-based 
assessments, 
include more 
complex questions 
and activities that 
allow learners to 
demonstrate what 
they learned in the 
curriculum. 



Designing curriculum-based assessments may seem easy. 

It can be if you use only simple vocabulary and recall compre- 
hension questions. Developing more complex questions (e.g., 
summary, inference, analysis) is time-consuming and often 
difficult; however, these questions are essential for building 
critical literacy skills. In addition, consider including activities 
that allow learners to demonstrate what they have learned in , 
the curriculum. For example, can learners apply skills covered 
in the curriculum to similar real-world situations? 
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Logs help learners 
record situations in 
which they use their 
skills outside the 
classroom. 



Logs and Checklists 

Adults can be encouraged to keep logs recording situations 
in which they use reading, writing, speaking, and math outside 
the classroom. You can suggest that they record what they 
read, how much they read, when they write, or how often they 
write. Learners might record situations in which they applied 
math principles or used reading skills to solve a practical 
problem. Over time, these logs will include more information 
as learners become accustomed to thinking about the role that 
literacy plays in their lives. You can use this information to 
document learners’ application of literacy skills in real-world 
contexts. 



Checklists provide 
more structure 
than logs and make 
it easier to analyze 
the collected 
irtformation. 



Checklists are more structured than logs, making it easier 
to analyze the information you are collecting. You can use 
existing checklists or develop new lists to meet your speciHc 
needs or learners’ goals. For example, existing checklists can 
be found in McGrail and Purdom (1992), McGrail and 
Schwartz (1993), and Auerbach (1992). 



Portfolios range 
from strictly 
personal to broad 
yet systematic 
and purposeful 
collections of 
student work. 



Portfolio Assessment 

You can select from a number of portfolio approaches, 
ranging from the strictly personal to broad yet systematic 
collections of student work. Portfolios, however, are not 
merely work folders. They are purpos^l collections of work 
that demonstrate progress or achievement in selected areas of 
study. Portfolios may be process-oriented (i.e., collection of 
writing samples in various stages of completion) or product- 
oriented (i.e., collection of finished projects). 



Portfolios can focus on one aspect of learners’ work (e.g., 
writing or math skills) or include several areas. They might 
focus on collecting information to document learners’ progress 
toward developing workplace competencies. For example, 
adults in a job preparation class may document their ability to 
be punctual by maintaining an ongoing record of their class 
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Portfolio contents 
may be 

skills-oriented 
or they may be 
more personal, 
such as records of 
how the program 
impacted the 
learners’ lives. 



arrival times; the list can be verified by project staff. On the 
other hand, a portfolio can be very personal, focusing on 
collecting evidence to document the impact of program partici- 
pation on the learner’s life. For example, parents may collect 
their children’s report cards showing improvements in atten- 
dance and grades as an example of the impact their participa- 
tion in an ABE class has had on their children’s lives. This 
portfolio has limited value for tracking the learner’s progress in 
developing basic skills; however, it may be invaluable to the 
individual as visual evidence of education’s value. 



Discuss with 
learners the 
specific 

requirements and 
ejqtectations for 
the portfolio 
assessment btf^ore 
beginning the 
process. 



Learners must be prepared to use portfolio assessment, 
since collecting and analyzing the work depends on their 
involvement in the process. Although you can, with the 
learner’s permission, add work to the portfolio, the primary 
responsibility lies with the learner. Therefore, it’s important to 
detail the specific requirements and expectations for the portfo- 
lio assessment before beginning the process. Set goals or 
outline the expected outcomes, identify the types of informa- 
tion that should be collected to demonstrate progress, and list 
criteria that will be used to evaluate the portfolio contents. As 
learners add material to the portfolio, they should include a 
brief explanation of why the material is important. This will 
provide an ongoing record of the learner’s thoughts about their 
work. Finally, work with the learner on a regular basis to 
review the portfolio contents and summarize progress toward 
the selected goals or outcomes. 



htformal 
assessments 
usually match 
the program’s 
curriculum or 
educational goals. 



Advantages and Limitations of Informal Assessments 
Informal assessments are valid for a program since they 
usually match the program’s curriculum or educational goals. 
Informal instraments are often more meaningful to learners 
since they can document steps or milestones toward achieving 
learners’ long term goals. Instructors and learners can see 
progress that might not be evident in standardized test scores. 
For example, a process portfolio allows learners to follow their 



They provide 
opportunities for 
instructors and 
learners to make 
joint decisions 
about individual 
education plans 
and goals. 



development from writing one sentence to a paragraph to 
several connected paragraphs. Or, a reading log might allow 
learners — and the instructor — to track increases in the amount 
and type of reading done outside of class. As a procedure 
rather than a formal test, informal assessments also can provide 
opportunities for instructors and learners to make joint deci- 
sions about individual educational plans and goals. 



Informal 
assessments may 
lack a clear purpose 
or may be poorly 
designed. They 
also may lack 
reliability. 



Although informal instruments are valid within a program, 
they have several drawbacks. Informal assessments may lack a 
clear purpose or may be poorly designed and organized, con- 
founding your efforts to collect meaningful information about 
learners’ skills. These instruments also may lack reliability, 
making it difficult to ensure that the scores are meaningful 
measures of improvement. 



Plan carefully 
before you begin 
developing irformal 
instruments. 



Designing Informal Assessments 

How can you develop effective informal assessments for 
your program? The disadvantages of informal assessments 
stem from their lack of clear purpose, meaningful design and 
organization, and reliability. To avoid these drawbacks, plan 
carefully before you begin developing the instrument. First, 
decide on the purpose of the assessment: 

• What are you trying to measure? 

• How will you use the information you collect? 

After you clarify the purpose of the assessment, identify spe- 
cific goals or expected outcomes: 

• What should learners be able to do? 

• How will they demonstrate the outcome? 

• What are the long-term and short-term goals to be 
reached? 

Finally, select criteria for measuring the learners’ level of 
success: 

• What milestones have learners reached that show progress 
toward their goals? 

• How will we define progress”? 




31 



2.9 



For example, the learner might be expected to write a cohesive 
paragraph consisting of a topic sentence and three supporting 
details. A milestone may be reached when the learner writes a 
coherent sentence on a single topic. A second milestone may 
be reached when the learner can write two coherent sentences 
on a single topic and a title (topic idea). 



Group decisions 
should address the 
purpose of the 
assessment, key 
outcomes or process 
to be measured, 
procedures for 
administration, and 
guidelines for 
scoring item and 
interpreting results. 



Reliable assessments provide meaningful information 
about learner progress. ABLE providers who develop infor- 
mal assessments should approach the process carefully. De- 
velopment should involve administrators, instructors, and, if 
feasible, learners as decisions are made concerning the pur- 
pose of the assessment, key outcomes or process to be mea- 
sured, procedures for administration, and guidelines for scor- 
ing items and interpreting results. Without discussion, for 
example, instructors may not agree on definitions of progress 
or success (e.g.. How is “little progress” or “much improved” 
defined? Do all staff agree on the meaning of an individual’s 
score of “4” on a checklist?). Lack of agreement on these 
issues will affect the instrument’s reliability. 



Irformal 
assessments may 
be standardized, 
although the 
process is 
time-consuming 
and costly. 



Some programs undertake standardizing their informal 
assessments. Although these efforts would improve the 
assessment’s reliability and, therefore, its usefulness for 
reporting purposes, the assessment may also lose some of the 
flexibility that initially made it effective. In addition, stan- 
dardizing informal measures is a time-consuming and expen- 
sive process; however, some programs choose to proceed (See 
Chapter 3, pages 3.4 - 3.7 on standardizing informal tests as 
well as Chapter 7 on selecting assessments for additional 
details). 



Summary 

Instructors and learners are most interested in individual 
progress. As a result, they often supplement standardized test 
data with information gathered through informal assessments. 



Informal instruments can include, for example, portfolio 
assessment, teacher-developed checklists and self-assessments, 
curriculum-based assessments, program-designed interviews, 
and commercially available rating sheets. Using informal 
assessments can involve instructors and learners in the assess- 
ment process, help learners set goals, guide instruction, docu- 
ment changes in a learner’s self-esteem, or track progress 
toward larger goals. These tools are valid within a program; 
however, the resulting data do not translate across programs 
nor is there evidence that the instruments are reliable. As a 
result, informal assessments are valuable tools but usually do 
not meet all requirements for program reporting and account- 
ability. 
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Chapter 3: How Can I Use 
Assessments to Document Learner 
Progress for Program Accountability? 


Program 

administrators want 
data to document 
the effectiveness of 
their programs. 


Introduction 

All stakeholders want information on program outcomes. 
Program evaluation and accountability, for example, are of 
particular concern to program administrators who must demon- 
strate to funding agents that students are effectively learning 
and meeting their goals through program participation. 


Funding agents 
want data that can 
be aggregated and 
used to describe the 
progress of groups 
of students. 


Funding agents are no longer satisfied with numerical 
reports of student attendance. They want data that can be 
aggregated and used to describe the progress of groups of 
students. Although they may be interested in individual 
progress, more inportatnt is information that can be summa- 
rized to allow comparisons of learner gains across programs. 
Their interest in data focuses on the general impact that adult 
basic and literacy education programs have on participants’ 
skills and, ultimately, in their lives. 


Instructors need 
objective evidence 
about their 
instructional 
effectiveness. 


Instructors also should be interested in program account- 
ability. Instructors are usually the single largest influence on 
students’ achievement. So, if a program is effective, it sug- 
gests that instructors are also effective. While instructors have 
a “gut” sense of their effectiveness fix)m student feedback and 
observation, they may also find it worthwhile to have objective 
evidence about instructional effectiveness. 


Program 
effectiveness must 
be demonstrated 
over time. 


Program effectiveness must be demonstrated over time. A 
single snapshot does not provide enough information to deter- 
mine whether students are learning in a particular instructional 
program. Most programs chart their progress over a number of 
years to determine program effectiveness. Some also conduct 
follow-up studies of program completers, as well as of 



dropouts, to identify factors influential in making the program 
successful for some individuals and to identify barriers to 
success for others. 



Standardized tests 
are superior to 
non-standardized 
assessments for 
program 
accountability. 
Group results rather 
than individual data 
are reported for 
accountability. 



What Types of Assessments are Appropriate for Program 
Evaluation and Accountability? 

Because standardized tests (given correct test administra- 
tion) ensure that results may be compared across programs and 
across the same program at different points in time, they are 
superior to non-standardized (i.e., informal) assessments for 
program evaluation and accountability. Non-standardized 
assessments do not permit such comparisons. In addition, 
group results, rather than individual data, are usually reported 
for program evaluation and accountability. Standardized tests 
are appropriate for these purposes, since test scores may be 
averaged and charted to display learning gains for a group of 
learners over time. 



Standardized 
tests may be 
norm-referenced, 
criterion-referenced, 
curriculum-based, or 
performance-based. 



Standardized tests may be norm-referenced, criterion- 
referenced, curriculum-based, or performance-based (or a 
combination of these types). The types of resulting informa- 
tion, however, vary depending on the type of assessment. 
When selecting an instrument, providers should consider both 
test validity and reliability — both are critical in choosing a test 
appropriate for your program. (See Chapter 7 for a detailed 
discussion of selecting assessments.) 



Most funding 
agencies prefer 
that student 
progress be 
measured by 
standardized, 
norm-referenced 
tests. 



Standardized tests that are norm-referenced compare 
learners to a previously established group of comparable 
individuals. For example, the Tests of Adult Basic Education 
(JABE) has been normed on three representative groups: 
learners in adult basic education programs, adults in voca- 
tional/technical programs, and adult/juvenile offenders in 
correctional settings. The TABE and the Test of Adult Literacy 
Skills (TALS) are both norm-referenced. They yield scaled 
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scores which provide a reliable indication of student growth. 
Most funding agencies prefer that student progress be mea- 
sured by standardized, norm-referenced assessments. 



Criterion-referenced 
assessments are 
expropriate 
for program 
accountability 
since they indicate 
students’ mastery 
of skills or 
competencies 



Curriculum-based 
assessments are 
usually not used for 
program evaluation 
and accountability. 



However, standardized criterion-referenced assessments, 
such as the Comprehensive Adult Student Assessment System 
(CASAS), are also appropriate for program evaluation and 
accountability. Because they indicate students’ mastery of 
specific skills or competencies, they may be an even better 
indicator of learners’ progress during instruction. 

Curriculum-based assessments, even if standardized, are 
usually not used for program evaluation and accountability 
since they measure learning resulting from a particular curricu- 
lum. Examples of curriculum-based assessments include tests 
included in the Challenger and Number Power series of text- 
books. For comparisons to be meaningful, all learners would 
have to use the same curriculum. The major exception is the 
GED Test which is a well-established curriculum-based assess- 
ment. 



Performance-based 
tests require 
the student to 
demonstrate skills 
or competencies. 



Performance-based measurement may be incorporated into 
one of the other types of assessment if a demonstration of 
learning is required. Performance-based assessments are often 
also criterion-referenced, requiring a demonstration of skill or 
competency mastery. The National External Degree Program 
(NEDP), for example, guides an adult through a series of 
activities to document their ability to apply literacy skills in 
typical adult situations, such as renting an apartment. The 
number of skills or competencies mastered before and after 
instruction is reported for program evaluation and accountabil- 
ity. Standardized, performance-based assessments, such as the 
NEDP, may be used for program evaluation and accountabil- 
ity. And, learners frequently find this type of assessment 
meaningful since they can see objective evidence of what they 
have learned. Performance-based assessments may also be 



ERIC 



38 



3.3 



particularly useful for non-native speakers and disabled indi- 
viduals who may not perform at their best on papCT-and-pencil 
standardized tests. 



Irtformal 
assessments are 
usually not 
appropriate for 
program 
accountability. 



Irformal tests are usually not appropriate for the purposes 
of program evaluation and accountability because they do not 
permit comparisons across programs. However, an informal 
assessment could become “standardized” by using standard 
directions for administration and, therefore, could provide 
useful information to a program. The limitation is that this 
assessment is unique to the program and, therefore, not useful 
to funding agents and other external audiences for accountabil- 
ity purposes. 



Programs may 
choose to 
standardize a 
locally developed 
assessment. 



How Can I Standardize a Locally Developed Assessment? 

Some programs prefer to use locally developed assessments 
that match their curricula. Most funding agencies, however, do 
not accept data from these assessments as the sole source of 
evidence about program effectiveness. Nevertheless, these 
assessments may have enough usefulness to a local program 
that an effort to standardize them is undertaken. Once the 
assessments are standardized, they may become useful to other 
programs with similar types of students and curricula. 



This first step is to 
create items that 
closely measure 
what is being taught 
in your curriculum. 



The process of stand?irdizing a locally developed assess- 
ment requires time and expertise. The first step is to create 
items that closely measure what is being taught in your curricu- 
lum^ If the items assess what you are teaching, your assess- 
ment will be valid for use in your program. If your assessment 
is criterion-referenced, then students who have been taught the 
assessed skills and presumably mastered them should score 
almost 100% on your assessment. 



Next, you should write standardized directions that all 
teachers and tutors will use in test administration. The 
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Write standardized 
directions for 
administration, 
scoring, and 
reporting data. 



Pilot test the 
instrument. Ask 
students about 
items they missed. 



assessment does not necessarily have to be timed, but it should 
be administered consistently. Procedures for scoring and 
reporting the data should also be standardized. 

The assessment should be carefully reviewed by colleagues 
for suggestions about the directions and wording for items as 
well as the format. After revisions, pilot test the instmment to 
determine directions or items that are unclear or poorly writ- 
ten. You can ask students about items that they miss, trying to 
determine why they answered as they did. 



Look for patterns 
in frequently 
missed items. 
Seek the help of 
a local university 
for data analysis 
and determining 
reliability. 



You should collect data, i.e., the items that are answered 
correctly and missed by each student, from a minimum of 
about 50 students that take the assessment. Even on paper you 
may begin to see patterns — for example, that students with a 
certain level of coirqjetence tend to miss given items. The 
formula for calculating reliability is influenced by the number 
of items in the assessment; therefore, a longer assessment is 
likely to be more reliable. One approach is to seek the help of 
a local university for data analysis in determining the reliabil- 
ity of the assessment. 



Computer programs 
can perform an item 
analysis to identify 
items that students 
with high total 
scores get right. 



You may also calculate reliability with computer programs 
in which an item analysis will tell you how closely each item 
correlated with the students’ total scores. If your test is a 
mastery or criterion-referenced test, you want items that 
students with high total scores get right. You may want to 
eliminate or reword items that students with high total scores 
miss. 



Collect data over an 
extended period of 
time to create local 
norms. 



To create local norms, you need to collect data over a 
period of time. Ultimately, you will be able to see that stu- 
dents with different mastery levels score differently on your 
assessment. At that point, the assessment can help you deter- 
mine the instruction needed by a new student as well as 
demonstrate program effectiveness when it is used after 
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instruction. It may also become useful to other programs with 
a similar curriculum. 



Qualitative data 
that have been 
appropriately 
quantified can 
be used to 
demonstrate 
program 
effectiveness. 



How Can I Use Qualitative Information for Program 
Accountability? 

The above effort of standardizing a locally developed 
assessment may not seem worth the trouble — which is why 
programs tend to use commercial standardized tests. You can, 
however, use qualitative information for program accountabil- 
ity if you quantify the data. The reason for quantifying qualita- 
tive data is to enhance reporting to funding agencies and other 
external audiences. Along with changes in standardized test 
scores, qualitative data which have been quantified can demon- 
strate that the changes are due to the impact of the program. 



Case studies of 
students in your 
program are an 
example of 
qualitative data that 
can be quantified. 



Sometimes programs may include case studies of selected 
clients who have, for example, moved from adult basic educa- 
tion classes to obtaining the GEP or high school diploma, and 
into better paying, satisfying jobs. These qualitative data, in 
the form of case studies, are important evidence of program 
effectiveness. However, case studies can also provide quanti- 
tative data which is useful in program accountability. 



Collect student 
outcome data, such 
as the numbers of 
students in your 
program who have 
found jobs or moved 
on to the next 
educational level. 



You may quantify these data by collecting evidence of 
program effectiveness over time. Some of these items are 
already reported to funding agencies in terms of numbers of 
students who have found jobs or moved to the next educational 
level. Some are less obvious and may require some special 
effort to collect, such as being able to help children with 
homework, being able to ask questions at work (in the case of 
ESL learners), and being able to use the transportation system. 



A suggested procedure is to interview students who are 
succeeding in their educational programs or who are recent 



Combine all the 
student outcomes 
in a list that is used 
to systematically 
and periodically 
collect data. 



program graduates. Determine how the program has affected 
their lives, both in important as well as small ways. Combine 
all the outcomes that the students name into a list, leaving 
space for others that may be named by other students. Then 
systematically and periodically collect this information from 
students through interviews and record it on a checklist. For 
each student you would have not only the results of assessment 
instruments but also a list of reported outcomes from participa- 
tion in the classes. 



Aggregate the 
outcome data to 
document program 
effectiveness. 



When you are asked to provide information for program 
accountability, you can aggregate the outcome data for your 
students, saying, for example, that 80 % of the students re- 
ported that they were better able to help their children with 
homework, and so forth. If you have access to a computer 
database program, such as FileMaker Pro, you can enter the 
data about each student in the database. You can search the 
database for particular outcomes, such as “child’s homework”. 
It will then list the number of times that the outcome was 
mentioned by your students. The quantitative data can also be 
accompanied by case studies which bring alive the accom- 
plishments of your students. 



Summary 

Most funding agencies prefer standardized, norm-refer- 
enced test data that document learner progress for program 
accountability. Other types of tests, when properly adminis- 
tered and reported, can also provide data for program account- 
ability. For example, a standardized and criterion-referenced 
assessment such as CASAS indicates students’ mastery of 
specific skills and can be a reliable indicator of learners’ 
progress during instruction. Programs may also standardize a 
locally developed test, although the process requires consider- 
able time and expertise. Finally, programs may also use 



qualitative data (e.g., case studies) that have been properly 
quantified to document program effectiveness. 
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Chapter 4: How Can I Use 
Assessments to Screen Learners 
and Guide and Evaluate Instruction? 



At the program 
level, test results 
are used to r^er 
learners for 
additional services, 
place them in 
appropriate 
instructional levels, 
and diagnose their 
strengths and 
weaknesses to guide 
instruction and 
assist learners 
in setting 
realistic goals. 



Introduction 

Test results should provide useful information to both 
instructors and learners. Unfortunately, learners — and, too 
often, instructors — ^perceive testing as a necessary evil, gather- 
ing essentially meaningless numbers for program reporting and 
accountability. While program accountability is a signiHcant 
purpose for assessment, it may seem unimportant to instructors 
and even less so to learners. However, test information is 
critical at the program level, because assessment is an essential 
component of instruction. At the program level, you should 
use test results to refer learners for additional services, place 
learners in appropriate educational settings and instructional 
levels, and diagnose learners’ strengths and weaknesses to 
guide instruction and assist learners in setting realistic goals. 



A quick, global 
assessment of basic 
skills during an 
initial interview can 
be used for r^erral 
or initial placement. 



Screening 

Adults entering an ABLE program should be given inune- 
diate, honest, and supportive feedback concerning the match 
between their goals and the program’s ability to help them 
achieve those goals. Information collected during an initial 
interview, therefore, should include a quick, global assessment 
of basic skills. A global assessment can accomplish one of 
two purposes: referral or initial placement. 



A global assessment 
of basic skills may 
be used to r^er the 
learner to a more 
appropriate pro- 
gram, if necessary, 
or for additional 
screening. 



A global assessment of basic skills can provide enough 
information to refer the adult to other or additional services, if 
necessary. For example, an adult may want to enroll in a GED 
preparation class. On completing a basic skills screening test, 
however, it appears that the adult would be better served in a 
literacy program. The GED program may encourage that adult 
to enroll in another community program if theirs does not 
provide basic literacy services. Results from the basic skills 
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screening, with other information collected during an intake 
interview (e.g., school experience, learning preferences), might 
also suggest that the learner be referred for additional screen- 
ing, such as vision, hearing, or learning disabilities screening 
(See Appendix C for sample checklists) or professional ser- 
vices. Professional services, for example, might diagnose and 
document a learner’s specific learning disability which, in 
turn, would allow that person to take a modified version of the 
GED test (See Chapter 6 for additional information on adults 
with learning difficulties). 



A global assessment 
is also used to 
place the learner 
in the appropriate 
level of instruction 
or materials. 



A second purpose for administering a global assessment is 
to place the adult in an appropriate setting for instruction. In 
this case, additional testing to determine specific strengths and 
weaknesses would be scheduled at a later time. For example, 
an adult may wish to begin an English as a Second Language 
instructional program immediately. If feasible, the ABLE 
provider would administer a screening instrument to determine 
whether the adult should be placed in a beginning, intermedi- 
ate, or advanced class. Providers who use specific instruc- 
tional materials also may use screening to place the learner in 
the appropriate level of the materials. For example, both New 
Reader’s Press Challenger series and Steck-Vaughn’s Reading 
for Today series include placement tools. [NOTE; These are 
only examples and not meant as an endorsement of either 
series.] 



Screening 
instruments provide 
a global view of 
learners’ skills 
rather than specific 
diagnostic 
irtformation. Also, 
they are not 
expropriate for 
general reporting 
purposes. 



Some assessment instruments — or sections of instru- 
ments — are appropriate for rapid screening of basic skills. 
These instruments may be standardized or informal; therefore, 
programs can choose instruments that meet their specific 
program design and assessment plans. When using screening 
instruments, remember that: 1) the results provide a global 
(or general) view of the learner’s basic skills rather than a 
diagnostic profile of strengths and weaknesses, 2) the 
instruments used are not appropriate for accountability or 
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general reporting purposes (i.e., They are either parts of tests or 
informal measures.), and 3) the instruments should be brief, 
taking between 5 and 20 minutes to administer. 


Programs may 
use standardized 
screening 
instruments, such 
as TABS Locator 
Test or SORT. 


Programs may use a standardized instrument, such as the 
locator test in the Tests of Adult Basic Education (TABE) or 
the screening instrument based on the Basic English Skills Test 
(BEST). The first is appropriate for native speakers of English, 
and the second is appropriate for non-native speakers. The 
Comprehensive Adult Student Assessment System (CASAS) 
also includes a placement instrument, the Life Skills Appraisal, 
as does the Adult Basic Learning Examination (ABLE). Some 
programs use the Slosson Oral Reading Test (SORT) or the 
Adult Learning Employment-Related Tasks (ALERT) as 
screening instruments. 


They also may use 
itrformal measures, 
such as word lists 
and cloze passages. 


Informal screening may include word lists within informal 
reading inventories, short pretests included with commercially 
available instructional materials, or teacher-made instruments 
(e.g., functional word lists; cloze passages; retell exercises with 
short, graded reading passages; learner-selected reading pas- 
sages). In each example, however, the results of the screening 
test will provide a global measure of basic skills to assist the 
provider in either referring learners to other or additional 
services or to place them in an appropriate learning environ- 
ment. 


Instructors may use 
standardized or 
itrformal tests to 
determine learners’ 
skills strengths 
and weaknesses 
to plan, guide, 
and evaluate 
instruction. 


Planning and Evaluating Instruction 

Instructors may use a variety of assessments to determine 
learners’ strengths and weaknesses in basic skills. These 
instruments may be standardized or informal, depending on the 
ABLE provider’s philosophy about and plan for learner assess- 
ment. For example, some programs may use only a standard- 
ized diagnostic basic skills test to place learners and to guide 
instruction. Other programs prefer to use informal assessments 
to determine specific instructional goals. Instructors should 



identify learners’ strengths and weaknesses soon after they 
enroll in the program and then work with the learners to 
develop an individual education plan (lEP). This diagnosis, 
however, provides only initial instructional goals. The instruc- 
tor should meet with the learner on a regular basis to revise the 
educational plan and identify alternative instructional strate- 
gies in areas where the learner has not made progress. 



Instructors should 
use assessments 
that provide 
diagnostic profiles 
for determining an 
individual's skill 
strengths and 
weaknesses. Survey 
tests typically do 
not provide 
irformation on 
specific types of 
skills errors. 



When using standardized tests to diagnose skills, instruc- 
tors should use instruments that provide diagnostic profiles 
rather than survey tests which, typically, do not provide infor- 
mation on specific types of skill errors. For example, the 
TABE Complete Battery provides a diagnostic profile of skills 
for each level of the test. On the other hand, the TABE Survey 
does not include a diagnostic profile of specific skills. Al- 
though skilled instructors can evaluate individual items to 
determine the types of skills missed on the test, the number of 
items is limited and may lead to an inacciuate evaluation of 
strengths and weaknesses. On the other hand, the BEST 
suggests that the instructor review specific sections of the 
assessment to determine the learners’ competence in language 
use. For example, the learner may be proficient at handling 
money but have difficulty telling time. Reviewing BEST 
subscores also may reveal that the learner is skilled in listening 
comprehension but has difficulty speaking. This information, 
then, can be used to plan instruction. 



Itrformal 
assessments also 
can be used to 
effectively guide 
instruction. 



You can also use informal assessments effectively to guide 
instruction (See Chapter 2 for descriptions of informal assess- 
ments). Instructors may use commercially available or 
teacher-constructed tests to identify specific skill strengths and 
weaknesses. Informal reading inventories, cloze passages, and 
retell exercises can provide detailed informaition about the 
learner’s word recognition and comprehension skills. Holistic 
scoring of writing samples can identify a learner’s skill in 
using grammatical structures and vocabulary as well as ability 



to produce a cohesive paragraph. Teacher-created math prob- 
lems can determine a learner’s skill in performing specific 
operations; word problems can identify the learner’s ability to 
read math-focused vocabulary and apply math operations. 

Some informal assessments incorporate activities that learners 
perform to demonstrate their ability to apply basic skills in 
specific contexts; these are performance-based assessments and 
can be standardized or informal. For example, learners might 
register to vote and bring a voter registration card and receipt 
to you to demonstrate that they had successfully registered and 
voted. Others may incorporate completing tasks at various 
skill levels. For example, a beginning reader might interpret 
simple written directions, and a more advanced reader might 
interpret multi-step written directions. 



Portfolios and 
observations may 
also provide useful 
irtformation about 
learners’ skill 
strengths and 
weaknesses as well 
as evidence of 
learner gains. 



You also may identify skill strengths and weaknesses by 
observing learners as they complete instructional tasks; these 
observations may be recorded in checklists or anecdotal 
records and reviewed to identify skill strengths and weak- 
nesses. In addition, many ABLE providers incorporate various 
types of portfolio assessment as a strategy for collecting infor- 
mation about learners’ strengths and weaknesses as well as 
evidence of progress. Analyzing portfolio contents and teacher 
observations can provide information that guides or modifies 
instruction. 



Instructors and 
learners should 
work together to 
use assessment 
results as a basis 
for planning 
instruction and 
setting 

personal goals. 



Setting Educational Goals 

In addition to guiding instruction, teachers should use 
assessment results to assist learners in setting realistic goals. 
Assessment is an integral part of instruction. Learners as well 
as instructors should view assessment results as a basis for 
planning instruction and setting personal goals. As such, 
assessment is a powerful strategy for engaging the learner in 
the educational process and building a more equitable relation- 
ship between the instmctor and learner. By engaging learners 
in the assessment process, the instructor relinquishes a meaure 
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of control over the educational plan. Learners then assume 
more responsibility for determining the educational plan and 
setting realistic goals based on test results. 



Adult learners 
may be reluctant 
to offer opinions 
or explore 
alternative goals. 



Adult learners may have had little or no experience dis- 
cussing test results with an instructor or being asked for their 
personal educational goals. As a result, they may be hesitant to 
offer their opinions or may not be willing to explore alternative 
goals. For example, an adult who wishes to obtain a GED may 
not want to change that goal even when faced with test scores 
indicating that the goal is unrealistic. Initially, you should 
redirect the discussion to specific skill areas. If the learner has 
stronger math skills than reading skills, the suggestion might 
be to focus on that area of strength, building related reading 
skills through a math-focused instractional plan. 



The instructor 
should act as an 
expert facilitator, 
guiding the learner 
yet encouraging 
responsibility for 
selecting 

educational goals. 



On the other hand, instructors using competency-based tests, 
such as CASAS, might discuss areas in which the learner has 
shown competence. The learner may wish to build greater 
competence in those areas but also work to improve skills in 
selected areas of weakness. For example, an adult may have 
strong skills in competencies listed under consumer economics 
but be weak in those related U) government and law. This 
learner’s immediate educational goals may be to expand die 
level of competence in selected areas of consumer economics 
but focus on building competence in understanding individual 
legal rights (one of the CASAS competencies in government 
and law). In each case, you should act as an expert facilitator, 
guiding the learner yet encouraging greater responsibility for 
selecting relevant and realistic educational goals. 



Summary 

Test results are critical at the program level because assess- 
ment is integral to instruction. Assessment information should 
be used to refer learners for additional services, make appropri- 
ate instructional placements, guide instruction, and assist 



O 




50 



Assessment 
information should 
be used to rrfer 
learners for 
additional services, 
make appropriate 
instructional 
placements, guuk 
instruction, and 
assist learners 
in setting 
realistic goals. 



learners in setting realistic goals. ABLE providers may attempt 
to locate one assessment instrument to fit all purposes; how- 
ever, this is not realistic. The “ultimate” test — one size fits 
all — does not yet exist. Rather, think clearly about the pur- 
poses of assessment and select instruments that best fit each 
purpose. For the purpose of screening learners and guiding 
instruction, perhaps one assessment is appropriate. It may be 
more realistic, however, to select several instruments that are 
valid (i.e.. The tests measure, as much as possible, the skills 
and knowledge taught in the program’s curriculum.) and that 
meet both the instructor’s and the learner’s informational 
needs. 



Tests 

See Appendix A and G. B. Jackson’s Measures for Adult 
Literacy Programs (available through Pennsylvania Literacy 
Resource Centers and Professional Development Centers) for 
additional test information. 
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Chapter 5: How Can I 
Assess Adults for Whom 
English is a Second Language (ESL)? 



ESL practitioners 
must address 
complex issues 
when developing an 
assessment plan. 



ESL assessment 
practices vary 
with the scope and 
mission of the 
program, the 
specific kinds of 
services provided, 
and the learners’ 
cultural and 
educational 
backgrounds and 
needs. 



Introduction 

No single test or type of test can address the complex 
assessment issues facing ESL practitioners. Commercially 
available, standardized tests, like the Basic English Skills Test 
(BEST) or CASAS, provide quantifiable information (such as a 
student’s listening comprehension proficiency), but they may 
not accurately reflect the student’s true communicative abilities 
in real-life situations. And while informal assessments, such as 
portfolios or presentations, may provide a more accurate 
picture of what learners actually can do and say in the English 
language, they often do not provide the kinds of data that 
administrators and funders need to demonstrate and compare 
program effectiveness. 

In addition, assessment practices in ESL programs vary 
with the vision and scope of the program, the specific kinds of 
services provided, and the learners’ cultural and educational 
backgrounds and needs. For example, assessment practices in 
an ESL workplace literacy program funded by an employer 
may be quite different from those in a local community-based 
literacy program that teaches basic communication skills for 
recent Russian immigrants who must learn both the English 
language and its alphabet. In another setting, two teachers 
might meet for the first time with 80 students who wish to 
enroll for an evening class. Intake assessment in this program 
would differ greatly from one in which an intake counselor 
interviews only a few students enrolling in the program. 



Despite this complexity and diversity, current research and 
practice suggest that effective ESL programs use a combination 
of commercially available, standardized tests as well as 
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Effective ESL 
programs use a 
combination of 
commercially 
available, 
standardized tests 
as well as 

program-developed 

irrformal 

assessments. 


program-developed informal assessments. This chapter does 
not identify or prescribe specific tests that should be included 
in an assessment plan since programs and their clientele vary 
so widely. Rather, it describes key issues to help ESL provid- 
ers develop or refine their assessment practices. When choos- 
ing assessments, ESL providers must consider the learners’ 
culmral and educational backgrounds and their levels of profi- 
ciency with the English language. They should also consider 
whether to test in the students’ native language. 


Consider the 
cultural and 
educational 
backgrounds of 
learners when 
choosing 
assessments. 


Learners’ Cultural and Educational Backgrounds 

Consider the cultural and educational backgrounds of your 
learners when choosing tests. If students are well educated in 
their first language, they can rely on their knowledge regarding 
oral and written language when learning a second language. 
Conversely, if a student’s native culmre does not have a written 
form of language or provide public educational opportunities, 
this student is not likely to perform well on a standardized 
achievement test 


Examine the 
development and 
content of the test in 
the manual. 


Consideration must also be given to the development and 
content of the test. Was it normed on ESL adults? Do test 
items require cultural background knowledge that may be 
unfamiliar to your learners? Minimally, students should be 
somewhat familiar with the testing procedures, content, and 
structure of the test items. Consult the test manual before 
selecting an instrument. Information on norming, test bias, and 
appropriateness for selected audiences should be included in 
the technical manual for any standardized test 


Choice of ESL 
assessments 
depends also on 
the language 
proficiency levels 
of the learners. 


Learners’ Proficiency with English 

Choice of assessments in an ESL program depends also on 
the language proficiency levels of the learners. Giving a 
complicated grammar test to a beginning student would not be 



An ESL beginner 
has little or no 
abilities in the 
English language. 



appropriate. Similarly, a simple verbal assessment would not 
provide an accurate measure of an advanced student’s ability to 
read and comprehend a complex technical manual. Effective 
ESL assessment decisions should consider what students can 
do with the language, not only what they know about the 
language. In addition, demonstrating that students have moved 
from one level to another is an appropriate assessment of 
progress. The following guide (used with permission of the 
author. Dr. Judith Rance-Roney) illustrates the various English 
language proficiency levels: 

Beginner 

A learner at this level may have little or no ability to speak, 
read, or write English and would be unable to function inde- 
pendently using the language. In listening, only short utter- 
ances, simple courtesy expressions, and main themes are 
comprehended. In writing, this student can copy, list, and label 
concrete terms and may be able to fill in simple autobiographi- 
cal information on forms. 



An advanced 
beginner has some 
ability to satisfy 
immediate 
communication 
needs in English. 



Advanced Beginner 

This learner will have some ability to operate in a limited 
capacity and to satisfy immediate needs in English. In listen- 
ing, this learner will be able to decipher the main idea of a 
dialog. A writer at this level will be able to produce simple 
paragraphs using familiar materials, and may read short pas- 
sages with general comprehension. 



Beginning- 
intermediate 
learners can 
understand easy 
questions and 
answers and 
hold simple 
conversations. 



Beginning-Intermediate 

A learner at this level will be able to satisfy siuvival needs 
and minimum courtesy requirements. The learner can under- 
stand easy questions and answers and hold simple face-to-face 
conversations. In reading, the learner will be able to read for 
information and identify supporting details. The learner can 
write letters and short compositions using simple grammatical 
structures. 
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Intermediate ESL 
learners are able 
to converse with 
native speakers 
and are understood 
when discussing 
familiar topics. 



Intermediate 

The full intermediate learner will be able to converse with 
native speakers and will be understood when discussing famil- 
iar topics. Listening may be selective, and the listener can 
identify mood and attitude of the speaker. Polite expressions 
are mastered at this level. In reading, the use of context clues 
and the skills of ski m ming and scanning are possible. Writers 
are able to take notes in class, and to write using common 
terms and vocabulary which is comprehensible to the a native 
speaker, but would be labeled simplistic. 



Advanced 
ESL learners 
communicate well 
at work and meet 
social demands 
of conversation. 



Advanced 

This learner can communicate well at work and can ad- 
equately satisfy the social demands of conversation with some 
sensitivity to both informal and formal language. Listening 
comprehension now can include abstract discussion and the 
details of everyday non-technical conversation. Abstract 
material can be comprehended in reading as well. Academic 
reading in history, cultural and moral issues, and politics is 
within this ability range. This writer can use both informal and 
fo rmal prose, paraphrase and summarize, and produce complex 
sentence structures with adequate accuracy. 



Programs must also 
consider whether to 
test in a studenfs 
native language. 



Testing in Native Languages 

Programs must also consider whether to test in a student’s 
native language when choosing assessments. This decision 
depends, in part, on the reason for testing. If a student is being 
tested on his knowledge of a particular content area (e.g., 
history or biology), then it is appropriate to test in the native 
language. For example, the results of the Spanish version of 
the GED Test should accurately assess a Spanish-speaking 
student’s subject knowledge. On the other hand, if you are 
assessing a student’s competency in English rather than his or 
her subject knowledge, the ^^g should be done in English. 
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Programs which 
follow a bilingual 
literacy model are 
more likely to 
test in a student’s 
native language. 



The decision to test in a student’s native language also 
depends on the scope and mission of the program. Programs 
which follow a bilingual literacy model are more likely to test 
in a student’s native language. The TABE, for example, is 
available in a Spanish version to measure the basic reading, 
math, and language skills of Spanish-speaking adults. Many 
programs also develop informal tests in the student’s native 
language and rely on bilingual staff for administering the 
assessments. 



Providers can 
collect essential 
background 
irformation during 
an intake interview. 



How Can I Assess Learners’ Backgrounds and Proficiency 
Levels? 

Intake and Placement 

Providers can collect essential information during an initial 
intake interview. A one-on-one interview helps staff gather 
relevant background information about the learner’s oral 
language proficiency, native language literacy and education, 
previous English language instruction, goals and reasons for 
enrollment, and learning styles or preferences. Sometimes a 
translator may be necessary. A questionnaire should be used to 
guide the interview. Students with higher levels of English 
proficiency can complete the questionnaire independently, thus 
providing additional information about their writing skills. 



Placement tests 
help to place 
students in the 
appropriate level 
of instruction. 



In addition, placement tests, such as the CASAS ESL 
Appraisal Test and the oral component of the BEST are used to 
place students in the appropriate level of instruction. They 
usually involve a short survey or locator test to initially place 
the student in a broad ability level, such as beginner, intermedi- 
ate, or advanced. This initial placement also helps to identify 
appropriate instructional materials. 



Diagnosis 

Based on the level identified by the locator test, additional 
testing is often administered to collect baseline data for measur- 
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Diagnostic testing 
provides irrformation 
about a learner’s 
strengths and 
weaknesses in 
specific skills areas. 



ing student progress and to obtain diagnostic information. 
Diagnostic testing provides information about an individual 
learner’s strengths and weaknesses in specitic skill areas, such 
as oral proticiency, grammatical knowledge, listening and 
reading comprehension, and writing competence. The BEST, 
for example, provides diagnostic information that can be used 
to develop individual education plans. An item by item review 
might indicate that an individual is able to correctly fill out 
forms but may have difficulty reading job advertisements in a 
newspaper. 



Standardized tests 
and iirformal tests 
can be used to 
measure student 
progress. 



Measuring Student Progress 

Periodic assessments of academic gain, achievement toward 
goals, and attainment of competence are needed to measure 
individual progress as well as group progress for program 
accountability. The same assessments used for pretesting 
should be used for interim and posttesting. In addition, it is 
often valuable to use alternative assessment procedures when 
measuring student progress for evaluating a student’s educa- 
tional plan. Audio tapes, writing samples, journals, and check- 
lists can be used effectively with ESL students. These more 
informal methods also provide simple yet relevant information 
for the student. 



Summary 

Assessment practices in ESL programs vary widely with the 
scope and vision of the program, the specific kinds of services 
provided, and the information needs of the various stakehold- 
ers. However, all ESL programs should select and use assess- 
ment instruments based on their learners’ cultural and educa- 
tional backgrounds, learners’ English language proficiency 
levels, and programmatic decisions regarding assessment in the 
students’ native languages. A complete assessment plan should 
include a variety of commercially available, standardized tests 
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as well as program-developed informal tests that address all 
purposes of assessment. 
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Chapter 6 



What Assessment Strategies Should I Use 
for Students with Learning Difficulties? 



Chapter 6: What Assessment Strategies 
Should I Use for Students with 
Learning Difficulties? 



Adult learners may 
have disabilities or 
learning difficulties 
that significantly 
c^ect their aca- 
demic performance 
or their testing 
abilities. 



Introduction 

ABLE practitioners walk a Bne line when testing students 
who come to their programs. On one hand, appropriate assess- 
ment procedures are essential for obtaining reliable, valid, and 
helpful test data. On the other hand, when students struggle 
visibly during testing, practitioners often feel compelled to 
make testing accommodations to ease their discomfort or 
fhistration. Some of these students may be uncomfortable in a 
formal testing setting; however, others may have disabilities or 
learning difficulties that significantly affect their academic 
performance or their testing abilities. 



Some adults may 
have diagnosed 
disabilities or 
learning difficulties; 

others may have 
learning difficulties 
that have not been 
diagnosed. 



Some adults arrive at the adult education program with a 
diagnosed disability. In these cases, specialists have conducted 
a series of tests to determine the nature of the adults’ disabili- 
ties. Information should be available concerning the diagnosis 
and appropriate accommodations. Other adults enrolling in 
ABLE programs also may have learning difficulties that have 
not been diagnosed, making it difficult to determine test 
accommodations and instructional strategies that might support 
their learning. 



Programs should 
fully document 
all testing 
accommodations. 



Testing accommodations are permissible, but they vary 
depending on the assessment being used and the specific 
learning difficulty. Whether students have diagnosed disabili- 
ties or suspected learning difficulties, programs should fully 
document all test accommodations. 



This chapter provides background information on the 
Americans with Disabilities Act of 1990, definitions of various 
learning difficulties, screening guidelines, and descriptions of 
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appropriate test accommodations and procedures. (See Appen- 
dix C also.) 


The ADA may have 
implications for 
some students in 
your program. 


What is the Americans with Disabilities Act? 

The Americans with Disabilities Act (ADA) of 1990 
prohibits discrimination against people with disabilities in the 
areas of public accommodations and services; this includes 
educational activities or schools receiving federal rinancial 
assistance. Therefore, if your program receives federal or state 
funding, it must comply with the ADA. This means that some 
students in your program may be eligible for modified testing 
procedures. 


Appropriate testing 
accommodations 
depend on the type 
of disability or 
learning difficulty. 


What are Disabilities? 

Experienced practitioners report an almost intuitive ability 
to identify which learners may have conditions that affect then- 
ability to learn. Although intuition is useful, it does little to 
address the question of providing appropriate assessment and 
instruction for the adults with suspected learning difficulties. 
Appropriate accommodations and instructional strategies 
depend on the type of learning difficulty; however, it is not 
easy to tell the difference between a learning disability and 
other emotional, physical, or cognitive conditions. 


Categories of 
disabilities include 
physical, develop- 
mental, and learn- 
ing disabilities, and 
attention deficits. 


In general, four broad categories of disabilities are recog- 
nized: 

♦ physical disabilities — ^visual/hearing/speech impairments, 
traumatic brain injury, orthopedic impairments (missing 
limbs or congenital birth defects), health impairments 
(asthma, diabetes, chronic or acute health problems) 

♦ developmental disabilities — ^mental retardation, autism, 
cerebral palsy 

♦ attention deficits — ^attention deficit disorder (ADD), atten- 
tion deficit hyperactivity disorder (ADHD) 



• learning disabilities — group of chronic learning conditions 

thought to be caused by distiubances of the central nervous 
system, including difficulty with listening, speaking, 
reading, writing, math, reasoning, and/or visual or auditory 
processing problems. 

ABLE programs often serve adults with a broad array of 
disabilities or learning difficulties as described above as well 
as adults with mental health conditions. These disabilities 
usually have a significant impact on the learners’ educational 
performance. The disabilities described above are chronic: 
they can often be treated, but they cannot be cured. 



Ask students about 
their educational 
ejq>eriences and 
arty disabilities 
for which they 
may have been 
diagnosed. Ask 
permission to 
obtain records from 
the appropriate 
specialist or 
agency. 



How Can I Determine if an Adult has Difficulty Learning? 
Intake Interview 

During intake, ask students about their educational experi- 
ences and any disabilities for which they may have been 
diagnosed. Younger adults may report diagnoses of learning 
disabilities or other learning difficulties that have been made 
through their school districts. Some adults will have diagnoses 
made through the Office of Vocational Rehabilitation, the 
Easter Seal Society, or other community agencies. Still others 
may have diagnoses made by a physician, psychiatrist, audi- 
ologist, or ophthamologisL Discuss with these students the 
potential impact that the disability may have on their educa- 
tional progress. Ask permission to obtain records from the 
appropriate specialist (psychologist, school, audiologist, etc.) 
for your program’s records. These records should include 
appropriate accommodations which will be helpful in design- 
ing an assessment and instmctional plan. 



Other adults may not have a diagnosed disability. These 
adults, however, may mention experiences that suggest 
learning difficulties. They may, for example; 

1. mention that they were in special education classes in 



O 

ERIC 



6 ^ 



6.3 



Adult learners 
may mention 
e^eriences that 
suggest they may 
have a learning 
difficulty although 
it has not 
been diagnosed. 



school. This information may or may not be helpful, since 
it doesn’t necessarily provide information about the type of 
disability they may have. This is especially true of adults 
who attended schools prior to testing that distinguished 
developmental disabilities from learning disabilities. Prior 
to this distinction, any student who had trouble learning 
might be placed in special education classes, regardless of 
their specific learning difficulty. 

2. mention accidents (e.g, head trauma) that seem to have 
affected their memory or their progress in school. 



Record information about potential learning difficulties in the 
adults’ folders. This information will inform decisions to 
screen for learning difficulties or refer for additional services. 
It also may help determine appropriate strategies for assess- 
ment and instruction. 



Providers servicing 
adults with the most 
limited literacy 
skills routinely 
screen for learning 
difficulties as 
part of the intake 
process. 



Screen for Disabilities 

Some ABLE providers report that they screen for learning 
difficulties as part of the intake process while other providers 
do not screen anyone. Typically, providers serving adults with 
the most limited literacy skills (i.e., adult literacy programs) 
screen as part of the intake process. Providers that focus on 
adult secondary education (i.e., GED preparation) may per- 
ceive no need for screening. Finally, others screen only after 
the learner has completed an initial assessment, a period of 
instruction and observation, and a second assessment. If the 
learner shows little or no progress, the instructor may then 
screen for suspected leanung difficulties. 



It is not the role of the ABLE practitioner to make medical 
or psychological diagnoses; however, it is to the benefrt of the 
student and the program if initial screening is conducted when 
a learning difficulty is suspected. Adults may have more than 
one type of learning difficulty; they are not mutually exclusive. 
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It is to the benefit of 
the student and the 
program if initial 
screening is 
conducted when a 
learning difficulty is 
suspected. 



In general, eliminate a purely physical cause, such as hearing 
or vision impairments, first Then, consider screening for 
conditions that are more difficult to identify, such as ADD, 
ADHD, or learning disabilities. Again, it is not the responsi- 
bility of ABLE practitioners to diagnose the condition; you 
only will screen for potential difficulties. Actual diagnosis 
must be done by a qualified specialist. See Appendix C for 
more information on screening. 



Make Appropriate Referrals 

If screening suggests that an individual has a learning 
difficulty, you should discuss seeking a formal diagnosis with 
the learner. There are reasons for and against doing this. 



If screening 
suggests that a 
student has a 
learning difficulty, 
discuss the 
advantages and 
disadvantages of 
seeking a formal 
diagnosis with 
the learner. 



Fowler and Scarborough (1993) mention three reasons 

why a diagnosis may be advantageous: 

1 . Adults with learning difficulties might benefit from 
alternative methods of instruction which may be identified 
through the diagnosis and identified as part of the 
diagnostician’s recommendations. 

2. Adults with a formal diagnosis can access specialized 
services such as occupational training or special testing 
accommodations in academic settings. 

3. A formal diagnosis — ^particularly of a learning disability 
(LD) — ^may raise the adult’s self-esteem. Individuals with 
LD exhibit average or above average intelligence but have 
difficulty in processing information. After years of 
difficulty in school, these adults are often relieved to find 
that they are not “stupid.” 



Some students 
do not seek 
formal testing. 



Although formal diagnosis may be worthwhile, some 
students prefer not to be labeled and do not seek formal 
testing. Also, some practitioners believe that formal testing is 
unnecessary since their instruction is individualized. In the 
end, the decision to seek formal diagnosis should be left to the 
learner. 
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If a student is in 
favor of formal 
testing, assist the 
learner in locating 
an appropriate 
diagnostic service. 



If the adult decides to seek a diagnosis, the ABLE program 
may assist in locating an appropriate diagnostic service. The 
Office of Vocational Rehabilitation, for example, may take a 
referral if the individual is also seeking employment. Commu- 
nity agencies such as the Easter Seal Society also may conduct 
evaluations. Some ABLE providers have developed relation- 
ships with local specialists (e.g., psychologists, optomotrists, 
audiologists) to conduct evaluations. 



If diagnostic 
services are not 
available, providers 
often rely on 
diagnostic skills 
tests or irformal 
teacher-made 
assessments. 



Unfortunately, it is more common to find that evaluation 
and diagnostic services are unavailable or inaccessible. In this 
case, ABLE providers often depend on standardized diagnostic 
skills tests or informal teacher-made assessments, information 
from the adult learner, and research on learning disabilities and 
learning differences to determine reasonable accommodations 
and appropriate instructional approaches. In any case, it is 
important for ABLE providers to document the types of ac- 
commodations made and instructional services provided for 
adults with suspected and with diagnosed learning difficulties. 



Always contact 
the publisher 
to determine 
allowable 
accommodations 
before 

administering any 
standardized test. 
Fully document arty 
accommodations 
you make. 



Accommodations for Assessment 

Always contact the publisher to determine allowable 
accommodations before administering any standardized test 
Test companies will have guidelines for using accommoda- 
tions, and the types of acceptable accommodations may vary 
considerably. For example, the GED Testing Service allows 
accommodations for individuals with disabilities who take the 
GED exam. For students to take advantage of the accommoda- 
tions, they must apply for a special administration of the exant 
A certified professional must document the disability and 
request specific accommodations. Physical disabilities, visual 
and hearing impairments, specific learning disabilities, 
psychological disabilities, and ADD/ADHD qualify a student 
for accommodations. Accommodations for taking the GED 
exam may include, for example. 
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Testing 

accommodations 
most often include 
changes in format 
or time limits. 



• time extensions 

• large print or Braille text 

• audiotaped administration 

• private testing room 

• readers or interpreters to read questions 

• scribes to record responses 

• frequent breaks 

Testing accommodations most often include changes in 
format or time limits. A change in testing format refers to the 
use of a different medium or method to present the same 
information without changing the content or the ability being 
tested. Braille, large print, or readers are examples of accept- 
able changes in test formats for visually impaired adults. 
Extending time limits on timed tests is an acceptable accommo- 
dation for adults with physical disabilities or certain learning 
disabilities. 



Consider the 
relationship 
between a test 
accommodation 
and the purpose of 
the assessment. 



In addition to these accommondations, consider the rela- 
tionship between a potential accommodation and the purpose of 
the assessment. For example, it is not appropriate to make 
accommodations for the skill that is being tested — e.g., don’t 
administer the test orally if you are testing reading ability, and 
don’t change time limits if you’re testing speed and accuracy. 



Summary 

The U. S. Department of Education has estimated that 
between 6% and 30% of the individuals enrolled in adult 
education programs have disabilities. These disabilities include 
examples from each of the four categories described earlier in 
this chapter. Adults whose disabilities affect educational 
performance are overrepresented in school dropouts, incarcer- 
ated individuals, mental health clients, and the unemployed. 
Also, they are underrepresented in vocational traiiung programs 
and in the workforce. The National Adult Literacy Survey of 



1993 reports that persons with disabilities scored at the lowest 
levels of literacy. 

Providing educational services for these adults is a chal- 
lenge: How should providers assess these adults’ literacy 
skills? What are the implications for instruction? What are the 
implications for documenting learner progress? Unfortunately, 
few answers exist. 

The provision of appropriate accomnKxlations is relatively 
clear when the learner’s disability has been professionally 
diagnosed. No such clarity exists for the majority of adults 
with suspected learning difficulties who are enrolled in ABLE 
programs. Providers must rely on screening instruments to 
identify potential difficulties, generally acceptable accommo- 
dations to determine assessment procedures, and diagnostic 
basic skills tests or informal teacher-made tests to determine 
appropriate instructional strategies. 

Generally, practitioners can base assessment procedures for 
adults with learning difficulties on several basic concepts: 

1. Involve the learners. Ask them for ideas on how they learn 
and on what types of accommodations are most useful. 

Help them evaluate and correct their errors. 

2. Assist the learners in setting short-term goals. Recognize 
their successes in meeting these goals. 

3. Discuss testing procedures with learners to make sure they 
understand what is expected. 

4. Discuss errors to determine whether the errors reflect a lack 
of knowledge or if they represent difficulties in processing 
information or performing. 

5. Document information about suspected or diagnosed 
disabilities as well as any accommodations made for 
testing. 
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Chapter 7: How Can I Select 
Appropriate Assessments? 



Many variables 
need to be 
considered when 
choosing tests for an 
assessment package. 



Introduction 

Making judgments about which tests will be used as part of 
an assessment package is not a simple task. Many variables 
need to be considered, including a test’s administrative and 
technical aspects, its purpose, content, and format (e.g., pencil 
and paper, computer-based, or other). All of these variables 
need to be judged in relation to their appropriateness for your 
assessment purposes, program structure, and learners. In 
addition, those who select and use tests must make decisions 
about whether to use commercially available tests or develop 
their own. 



Read the test 
description and 
then take the test 
to determine 
on a superficial 
level if it meets 
your purpose. 



Experts recommend that you first read the test description 
in the manual to determine if the test meets your assessment 
piupose. Then, take the test to determine if the content is 
appropriate. If these preliminary activities indicate that the test 
seems relevant to your needs, a more thorough examination of 
the test and manual is recommended. 



A well-written 
manual will 
describe complex, 
technical 

irrformation simply 
and clearly, without 
confusing jargon. 



This chapter describes the information that should be 
included in a manual for a commercially developed test. It also 
includes a checklist to help you examine a test to decide if it is 
appropriate for your assessment needs. In general, the techni- 
cal manual should describe the test, its purpose, and the in- 
tended audience. It should describe what will be tested and 
how, and should include a detailed discussion of how the 
results should be scored, interpreted, and used. A good, well- 
organized technical manual will provide this information 
simply and clearly without confusing jargon. 



The following sections provide in-depth infonnation about 
what the manual should tell you about each aspect of test 
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selection. Some sections may be more or less important to 
you, depending on your purpose. For example, if you are 
e xaminin g commercially available informal measures, such as 
reading inventories or checklists, then the section on technical 
aspects may not be relevant to your needs. 



The technical 
manual should 
thoroughly explain 
how the test 
is administered, 
scored, and 
interpreted. 



Administrative Aspects 

The technical manual should help you answer the follow- 
ing questions: Who should administer the test? Is it adminis- 
tered to individuals, a group, or either? What is the administra- 
tion time? How is it scored? How easy and objective is the 
scoring? Is training required for administration or scoring; if 
yes, is the training extensive? How can test results be used in 
the program? 



The norming group 
should represent the 
audience for whom 
the test is intended, 
and ideally, your 
student population. 



Technical Aspects 
Norming Data 

If the test is norm-referenced, the manual should describe 
the norming group. It should include information about the 
norming group’s age, gender, social class, race or ethnicity, 
schooling, and geographical location. The norming group 
should be large, and, more importantly, it should represent the 
group for whom the test is intended, and ideally, your group. 
For example, if you are searching for an ESL test, the norming 
group should include non-native speakers. 



YaiiiUiy 

Validity means the test measures what it says it measures. 
Content validity is established by expert review. For example, 
developers of the GED test use experts to select test items that 
correspond to the high school curriculum. 



Predictive validity 
indicates how well 
the test scores 
correlate with other 
desired outcomes. 
Concurrent validity 
tells you how well 
a test correlates 
with some other 
measure of the 
skills. 



The technical manual often reports validity in terms of how 
well the test correlates to some other measure of the skills. 

This is a measure of the test’s concurrent validity. The manual 
may also discuss the test’s predictive validity. Predictive 
validity indicates how well the test scores correlate with other 
desired outcomes. For example, predictive validity is often 
used by workplace literacy programs to predict whether work- 
ers will be able to transfer skills learned in the program back to 
their jobs. Validity es tima tes (whether concurrent or predic- 
tive) of .5 and below indicate that the test has low validity; 
estimates of .8 and above indicate high validity. 



Examine a test 
irformally to 
determine if its 
content is valid for 
your program and 
assessment needs. 



Practitioners also should examine the test’s validity infor- 
mally to determine how valid it will be for their program and 
assessment needs. Do the test questions match the content of 
the program? Does the test provide an adequate sample of the 
various aspects of yotu* curriculum? Does the test measme the 
skills or abilities it is designed to measure? Answering these 
questions will help to determine whether the content of the test 
is valid for yom program. 



Reliability means a 
test produces 
consistent results 
every time it is 
administered. 



Reliability 

Reliability means a test produces consistent results every 
time it is administered. It relates to how accurately a test is 
able to find the “true score’’ for a person. A true score is 
simply what es^ level of ability or achievement the person has. 
When we give tests, we see “observed scores.’’ Numerous 
things (e.g., fatigue, poorly worded test items, or too few test 
items) will keep the observed score from being the same as the 
true score. A reliable test will reduce the gap between 
observed scores and true scores. This gap between true and 
observed scores is the standard error of measurement. The 
smaller the error, the better the reliability. 
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7.3 



The importance 
of test reliability 
depends on how 
you plan to use 
the test results. 


Reliability is usually reported on a scale from zero to one. 
A value of zero means zero degree of consistency between 
observed and true scores. A value of one means that the 
observed scores and the true scores are the same. A test with a 
high reliability (.8 or above) is important when you are making 
decisions about individuals, such as using test results to decide 
if a person will move on to a higher level or new program. If 
you are averaging test scores to report group results, however, 
a more moderate reliability estimate (about .6 to .8) is accept- 
able. 

Manuals may report different types of reliability. In gen- 
eral, reliability is measured and reported in terms of time, 
items, and scorers. 


Test-retest 
reliability provides 
assurance that the 
test scores are 
consistent over time. 


• Tests should have stability over time. One way to assess this 
stability is test-retest reliability. That is, if students are given 
a test and an alternate form two weeks later and their scores 
are vastly different, neither set of scores are reliable. Test- 
retest reliability provides assurance that the scores received 
on the test are consistent over time. 


The results on one 
form of a test should 
correlate highly 
with the results on 
another form of the 
same test. 


• Tests with alternate forms must have equivalent items’, that is, 
the results on one form should correlate highly with the 
results on another form and should produce comparable 
means and standard deviations. In general, tests with more 
items tend to more reliable. 


Internal consistency 
means how well 
item scores 
correlate with other 
item scores 
within a test. 


• Reliability is also measured according to internal consistency 
or how well item scores correlate with other item scores. The 
higher the average of all these item to item scores, the greater 
the internal consistency reliability of the test. Internal consis- 
tency is important because if test items do not measure the 
same thing, they should be reported as separate scores on 
sub-tests. Internal consistency is usually reported in one of 
two ways: split half reliability or KR-20 reliability. Split half 
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reliability is Agiired out by correlating people’s scores on two 
halves of a test. KR-20 is a more precise measure of internal 
consistency because it is based on the fact that tests can be 
halved in many ways. KR-20 is basically an average of 
different split-half reliability calculations. 


Inter-rater 
reliability means 
how closely multiple 
scorers score a test 
in the same way. 


• Scorer reliability, or inter-rater reliability, means how closely 
multiple scorers score a test in the same way. It is important 
when scoring is judgmental. For example, inter-rater 
reliability is more important when selecting tests that assess 
writing skills by essay than it would be for tests that assess 
writing knowledge by multiple choice questions. Well- 
trained scorers help to raise scorer reliability. 


The manual should 
describe what stalls 
are assessed and 
how the results 
can be used. 


Purpose and Content 

The manual should describe what skills are assessed and 
how the results can be used for screening and placement, 
diagnosis, and/or measuring gain. It also should provide a 
description of how the test items were chosen and developed. 


Examine the 
content of the test 
from the students’ 
point of view. 


Examine the test items from the students’ point of view. 

Is the text layout readable and pleasing to the eye? Are the 
graphics simple and clear? Are the directions written or given 
orally? If they are written, are they written in plain English? 
Will a student’s reading comprehension interfere with accurate 
measurement? Does the manual state whether written direc- 
tions can be given orally? Do the directions provide illustra- 
tive, sample items? If the test includes an answer sheet, is it 
likely to be confusing for your students? 


The manual 
should report 
how the test was 
evaluated for 
test bias. 


Tests should not be offensive with regard to race, gender, 
age, or ethnic origin. The manual should report how the test 
was evaluated for test bias. For example, tests may be 
screened for offensive items by experts as well as representa- 



tives of the testing group. Or, they may have been subjected to 
statistical procedures, such as obtaining a bias index. Statisti- 
cal procedures, such as the bias index, involve predicting how 
well a certain group will do on test items, administering the test 
to the target group, then eliminating those items identified with 
a high chance of being biased in favor of or against a certain 
group. 



Consider the test 
format and its 
appropriateness 
for your program 
and resources. 



Format 

Finally, consider the format of the test. If it is a paper and 
pencil test, is it durable? Will you need to replace test booklets 
frequently? If the test is computer-based, do you have the 
required hardware? Do your students have the computer skills 
necessary to yield accurate test results on a computer-based 
test? Can the test be customized (i.e., can you add your own 
test items)? If the test is performance-based, do you have 
the necessary resources Gii^e space) that are necessary for 
demonstrating competency? All of these factors concerning 
the format of the test must be considered when selecting 
assessments. 



Summary 

Programs must consider a test’s administrative and 
technical aspects, its audience, purpose, content, and format 
when choosing assessments. All of these factors must be 
judged in relation to their appropriateness for your assessment 
purposes, program structure, and learners. Cormnercially 
available tests should have a technical manual that clearly 
explains all aspects of the test 
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7.7 



Assessment Selection Checklist 



Test Name 

Publisher 

□ Performance-based 

□ Informal (e.g., interview, 
checklist, portfolio, etc.) 



Type of Test 
I I Norm-referenced 

□ Criterion-referenced 

□ Curriculum-based 



Administrative Aspects 


Yes 


No 


Comments 


Training required 








Reasonable administration time 








Individual/group/either 








Alternate forms available 








Ease of scoring 








ScOTes useable by students 








Clarity of manual 








Cost 









Purpose (Can it be used for:) 


Yes 


No 


Comments 


Screening 








Placement 








Diagnosis 








Progress 








Program accountability 
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Technical Aspects 


Yes 


No 


Comments 


Norming data provided 








Validity d^ta provided 








Reliability data provided 









Content 


Yes 


No 


Comments 


Clear directions 








Age-appropriate 








Culturally-appropriate 








Relevent to your program 








List skills assessed Comments 









Format 


Yes 


No 


Comments 


Paper and pencil 








Computer-based 








Oral 








Other 









Additional comments: 

Strengths and weaknesses: 

Ways this assessment could be used in your program: 



Chapter 8 



How Can I Provide a Supportive Testing Environment? 
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Chapter 8: How Can I Provide a 
Supportive Testing Environment? 



Test anxiety 
affects many adult 
learners and must 
be addressed in 
ABLE programs. 



Introduction 

Test anxiety is a common phenomenon in adult education 
settings. Adult learners may become visibly uncomfortable 
when they are placed in a testing situation, especially learners 
who have avoided returning to the educational setting because 
of unpleasant associations from their past schooling and 
testing experiences. They may sweat or sigh audibly diuing 
testing; their hands may shake or they may frdget nervously. 
Many have not taken a test in years. Some may never return 
to the program if they are tested too soon or too rigorously. 



Adult educators 
must provide a 
supportive testing 
environment 
without 

compromising the 
integrity of the test. 



Adult educators must provide a testing atmosphere that is 
comfortable and reassuring, yet one that yields an accurate 
picture of the adult learners’ educational achievements, 
strengths, and weaknesses. This chapter describes factors 
within the learner (such as test anxiety) and the program (such 
as scheduling) that affect the testing situation. It also provides 
strategies to address these factors in ways that will help to 
provide a supportive testing environment without compromis- 
ing the reliability and integrity of the assessments. 



Test anxiety is both 
a physical and an 
emotional response. 



Address Learner Factors 
Test Anxiety 

Test anxiety is both a physical and emotional response that 
may affect a learner’s ability to concentrate and perform well 
on a test. Physical responses may include sweating, shaking, 
or trembling. The student may also experience an upset 
stomach or an accelerated heart rate. Emotional responses 
may include a feeling of panic, negative self-talk (/ can’t do 
this, or I hate taking tests), and an exaggerated dread or even 
avoidance of the testing situation. 
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8.1 



Test anxiety 
has its roots in 
past experiences. 



Practitioners 
should be sensitive 
to the signs of 
test anxiety. 



Address 
test anxiety and 
test-taking 
strategies soon 
cfter students 
are enrolled. 



For many adult learners, test anxiety has its roots in past 
experiences. Adult learners may have a history of repeated 
failures (either real or perceived) in school. They may have 
received negative messages from significant others, such as 
teachers, peers, or parents, about their ability to do well on tests 
or in school, and their self-esteem may be low. Some ESL 
students may have had little formal education in their native 
countries and may be unfamiliar with the testing situation itself. 
And, many adult learners may simply have never learned 
specific test-taking strategies to ease their anxieties. Some or 
all of these factors combine in varying degrees to create an 
unrealistic perception of the testing situation. 

To become more aware of test anxiety, listen closely to 
what learners say during intake interviews. Do they make 
negative or self-deprecating comments? Watch learners during 
and after testing. Do they become visibly uncomfortable? 
Above all, trast your instincts and knowledge of learners’ 
performance in the program. For example, do some learners do 
well in class or in tutoring sessions but score far below what 
you would expect on a test? If you suspect that test anxiety is 
affecting an individual or a group of learners, address it directly 
with the strategies in the following sections. 



Test-Taking Strategies 

Ideally, an ABLE program will provide opportunities 
immediately for students to learn how to be successful in the 
testing situation. If feasible, a workshop or one or two class 
sessions should address test anxiety and test-taking strategies. 
These opportunities are especially important for students who 
have been out of school for an extended period or who may 
have severe test anxiety. 

Discuss test anxiety; talk about learners’ past testing experi- 
ences and how they were negative or positive. Discuss how 



Teach students 
strategies to use 
before, during, and 
(rfter taking tests. 



testing is used in your program and why. Help students 
understand that taking tests can be a positive and informative 
experience. Explain that the testing information will help 
them succeed in the program. Share the following test-taking 
strategies with your students: 

• Get plenty of sleep before a test. Arrive early and be 
prepared. 

• Read the directions carefully. Know the time limit. 

• Don’t spend a lot of time on difficult questions. Answer 
easier questions first. Mark questions difficult questions 
and come back to them later if you have time. 

• Qearly fill in the circle for each answer on a “bubble 
sheet.” 

• Look over answers to see if they make sense. 

• For true-false questions, if any part of the statement is 
false, the whole question if false. Look for qualifiers like 
“all, most, or never.” 

• For multiple choice questions, try to answer the question 
before looking at the choices. Eliminate obvious wrong 
answers first. 

• For essay questions, look for key words like compare, 
describe, evaluate, or summarize that give clues for how to 
organize and write the essay. 

• After the test, ask the instructor for more information on 
incorrect items. 

• Practice test-taking! Use exercises, reviews, and practice 
tests. 



Strategies to Reduce Test Anxiety 

Some programs may not be able to teach test-taking 
strategies prior to testing due to time constraints. However, all 
programs can provide a supportive testing environment that 
will help learners overcome or ease their testing anxieties. 

The following strategies can be used before, during and after 
testing: 
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8.3 



All programs 
can provide a 
supportive testing 
environment 
before, during, 
and cfter testing. 



• Avoid the word “test,” especially if your students seem 
overly sensitive to the testing situation. For example, you 
might say, “I’d like you to answer these questions and 
solve these problems so we can decide what skills to work 
on. We’ll go over the answers together. This will help 
you be more successful in the program.” 

• Explain the purpose of the test in clear, simple language. 
Make sure everyone understands. 

• Ask if students have any anxieties or worries about taking 
the test. Discuss test anxiety and its symptoms. 

• Provide relaxation and positive visualization exercises, 
such as having everyone straighten their spine and take a 
deep breath. Have them close their eyes and visualize 
success. 

• Let students ask questions before they begin. 

• Allow students to write in the test booklets if they can’t use 
the answer sheets, or if they have vision or perception 
problems. 

• Watch learners while they’re taking the test. Approach 
anyone who appears upset or frustrated and offer reassur- 
ing comments. In severe cases, you may want to allow the 
individual to stop taking the test. You can reschedule the 
test for another time. 

• Allow students to review tests after they are analyzed and 
discuss the kinds of errors that were made. Schedule 
private meetings to review individual test results. 



Address Program Factors 

Scheduling Assessments 

New and experienced practitioners raise important issues 
about the timing of assessments. When should we screen 
learners for initial placement or referral? When is the best 
time to conduct diagnostic testing for instructional planning? 
How often should we test to track learner progress either for 
updating the instructional plan or to collect data for program 
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Assessment 
schedules should be 
based on a 
program’s 
structure, its 
assessment plan, 
and the needs of all 
the stakeholders. 



accountability? No assessment schedule will meet the needs of 
all ABLE programs. Providers must design an appropriate 
schedule based on the program’s structure, its assessment plan, 
and the needs of their particular stakeholders. Research on 
assessment practices, however, points to several strategies for 
designing an effective schedule for assessment. 



Establish consistent 
guidelines based on 
the purposes for 
assessment. 



Establish consistent guidelines based on the purposes for 
assessment. It’s important that all program staff understand 
these guidelines and comply with them. Program adminis- 
trators should provide all counseling and instructional staff 
with an orientation that includes a review of these guide- 
lines to ensure that assessment data are collected in a consis- 
tent manner. Staff should be aware of the assessments their 
program uses for screening, for diagnosing skill strengths 
and weaknesses, and for documenting learner progress. 

They should understand and use appropriate procedures for 
administering standardized and informal tests and for 
recording data. 



Involve learners 
in the 

assessment process. 



Involve learners in the assessment process. Instructors and 
students should view assessment as part of the learning 
process. Instructors and students should meet regularly to 
discuss assessment results and to plan or revise educational 
plans. Effective assessment should help learners identify 
learning goals and help instructors and learners plan mean- 
ingful instruction. (Students are more likely to recognize 
the importance of testing when they play an active role in 
using the information for instructional planning.) 



Review testing 
procedures to 
evaluate their 
appropriateness 
and effectiveness. 



Review testing procedures to evaluate their appropriateness 
and effectiveness. Program planning is an ongoing process. 
ABLE providers should consider review and evaluation of 
their program’s various components as a process of renewal 
and improvement. Both internal (e.g., staff turnover) and 
external factors (e.g., funding requirements) impact a 
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8.5 



program’s ability to meet the needs of adult learners in their 
programs. Therefore, it’s recommended that program 
review their testing procedures on an annual basis. 



When students 
arrive at your 
program, they 
should experience a 
safe and supportive 
environment. 



Intake and Screening 

When students arrive at your program for the first time, 
they should experience a safe and supportive environment. 
Avoid administering lengthy diagnostic or survey tests during 
this first visit. You should, however, describe the program, 
explaining that you will use various assessments to help them 
identify weak skills areas as well as strengths on which to 
build. You may want to administer a brief screening instru- 
ment if the adult seems willing to comply. This will give you a 
general sense of the individual’s skill level for initial place- 
ment. This screening may be standardized (e.g., TABE Locator 
Test) or informal (e.g., brief interview or cloze passages). 
Review the results of this screening immediately with the 
learner or at a follow-up meeting. 



Diagnostics tests 
and tests to 
obtain baseline 
measures for 
program 
accountability 
should be 
administered after 
students are 
conrfortable in 
the program. 



Diagnostic Tests and Testing for Accountability Purposes 
Programs should administer tests to diagnose skills 
strengths and weaknesses. In some cases, the diagnostic tests 
also may be used for accountability purposes. In other cases 
(e.g., the diagnostic tests are informal), a second test — one that 
allows for the collection of data that can be summarized — 
must be given to measure learner progress. These tests should 
be adnunistered after the adult has completed an orientation 
period, ideally including a class or workshop on reducing test 
anxiety. Some programs require that students be tested within 
a certain number of hours after eiuollment. It’s recommended 
that you test at or near the end of this period. For example, if 
you must test within the first 12 hours of eiuollment, test at the 
1 1th hour rather than earlier. In addition, many standardized — 
and some informal — ^tests are lengthy; however, it’s important 
that students not feel over-tested. Therefore, don’t test for 
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more than two hours in one sitting. For longer tests, break the 
testing sessions into smaller time chunks. The time between 
administering a pretest and posttest will vary, depending on the 
assessment. Generally, however, post test after the learner has 
completed a minimum of 30 hours of instmction. Informal 
assessments may be administered more often if the results can 
be used to improve instmction or revise learner goals. 



Summary 

Adult educators can provide a supportive, yet precise, 
testing environment by addressing both learner and program 
factors that affect the testing situation. For example, discuss- 
ing testing procedures, test anxiety, and test-taking strategies 
soon after students are enrolled provides a supportive testing 
environment for learners. Address program factors by estab- 
lishing consistent guidelines for testing based on the purposes 
for assessment, involving learners in the assessment process, 
and frequently reviewing assessment practices. These strate- 
gies will help to provide a comfortable and reassuring testing 
environment, yet one that yields an accurate picture of learners’ 
educational achievements and abilities. 
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Appendix A 

j 

Test Summaries Information 



The test summaries in this appendix supplement G. Jackson’s Measures for Adult Literacy 
Programs. Jackson’s book has been included with copies of this guide sent to the 
Pennsylvania’s Professional Development Centers (PDCs) and State Literacy Resource 
Centers (SLRCs) — Advance and the Western Pennsylvania Literacy Resource Center. 
Individuals may order copies of Jackson’s book by contacting The ERIC Qearinghouse on 
Tests, Measurement, and Evaluation, American Institutes for Research, 3333 K Street NW, 
Washington, DC 20007. Individuals in Pennsylvania may borrow a copy of the guide and 
book by contacting the PDCs or SLRCs. Copies of the guide may be purchased by 
contacting the Institute for the Study of Adult Literacy, Penn State University, 102 Rackley 
Bldg., University Park, PA 16802-3202. 



BEST COPY AVAILABLE 



A Day In The Life....Assessment 

Assessment Software Program 



Author & Date 

Developed by the Institute for the Study of Adult Literacy at Penn State, in partnership with 
Curriculum Associates (1994). 

Purpose 

To assess basic skills through interactive job-related scenarios. 

Description 

Using a functional context, competency-based approach, A Day in the Life. .Assessment 
measiues basic reading, writing, math, and problem solving skills through job-related activi- 
ties. Learners are testra, scored, and evaluated on skills that are critical to obtain or progress 
in entry-level jobs. The skills are keyed to the CASAS list of basic skills. Six career modules 
are av^able: Food Services, Health, Maintenance, Retail, Clerical, and Customer Service, 
(^estions are open-ended, ftU-in, and multiple-choice style. Intended for learners reading at 
about a third to fifth g^e level. 

Administration 
Computer-based; untimed. 

Alternate Forms 
None. 

Reliability 

Modules had a reliability of about .8 in the field test sample. 

Validity 

Validity was established by basing test items on detailed literacy task analyses in the target 
job areas. Content experts reviewed and validated items in all job areas. 

Scoring & Interpretation 

A Day in the Life.. Assessment contains a management system that scores the assessment and 
iden^es learners’ problem areas using the CASAS skills framework. The scores are re- 
corded on personal data disks from which detailed reports can be viewed on the computer 
screen and/or printed. 

Comments 

Available in both Macintosh and IBM/PC Windows formats. Please check for hardware and 
software compatibility. Will be available on CD-ROM in 1997. 

Availability ^ 

Curriculum Associates, Inc. ' 

P.O. Box 2001 

North Billerica, MA 01862-0901 
(800) 225-0248 

Price 

Complete set for assessment: $499.90 (single user), $1,129.90 (5-pack). Individual modules 
for assessment: $199.90 each (single user), $449.90 (5-pack). Career Modules: $129.90 
each (single user), $299.90 (5-pack of same module). Other combinations and discounts 
available. 

Source 

Curriculum Associates descriptive and promotional material. 
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BEST COPY AVAILABLE ^ ^ 



Adult Performance Indicator (API) 



Author & Date 

Kent L. Copeman and Karen A Ribaichik (1986). 

Purpose 

To measure paragraph comprehension and vocabulary. 

Description 

Standardized reading inventory serves as a tool for general placement decisions. Measures 
grade 2-9 reading level equivalents. Not intended as a measure of learning gains. The test is 
comprised of 50 vocabulary questions and 50 reading comprehension questions, all multiple 
choice. 

Administration 

This 20 minute, silent reading test can be administered to individuals or groups. The student 
or students read and silently respond to questions. Timed, with a five-minute limit for the 
vocabulary section and a 20 minute limit for the comprehension section. 

Alternate Forms 

Available in two forms. Form A and Form B. Non-consumable format. 

Reliability 
Not available. 

Validity 

API scores correlate highly (.88) with Nelson Reading Test. 

Scoring & Interpretation 

Measures reading levels from grade equivalencies 2-9. This test can be used for grade level 
placement in ABE, GED, ESL, special needs, and high school completion classes. 

Availability 
Hadley Press 
Box 147 

Hadley, MI 48440 
(810) 797-4407 

Price 

The price of $43.95 includes 20 test booklets, 1 blackline answer sheet, 1 blackline answer 
key and 1 examiner’s manual. Testing kits and other materials available. 

Source 

Hadley Press promotional material. 
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Adult Learning Em^to^ment Related Tisks 



Author & Date 

Meryl Lazar, Ph.D. and Rita M. Bean, Ph.D., University of Pittsburgh (1991). 

Purpose 

To screen candidates for workplace literacy programs. 

Description 

The ALERT test has three components: silent reading for comprehension of a general exposi- 
tory package, math computation, and a writing sample. The AIERT reading test is a maze 
comprehension task in which three word choices are provided for each deleted word. The 
math section involves completion ofcomputational basic math problems in a two minute 
period. For the writing assessment, students are given a topic and asked to think about it for a 
minute and to write on it for three minutes. The focus is on the number of words generated, 
not on the correcmess of the mechanics. 

Administration 

Individual. Timing varies with each test component. 

Alternate Forms 

Forms for pre- and posttesting available. 

Reliability 

Information not available. 

Validity 

According to the report describing the development of the ALERT, the test authors were 
“unable to validate empirically” the assessment They state, however, that content validity 
was high, because the test was an outgrowth of an existing successful curricula. 

Scoring & Interpretation 

Scored on site. According to the authors, the results of the ALERT should be fully integrated 
into the curriculum. For instance, ALERT results should be reviewed as an aid to creating 
program goals and used to rate program effectiveness as well as student learning. 

Availability 
Dr. Rita M. Bean, 

University of Pittsburgh 
5N25 Forbes Quadrangle 
Pittsburgh, PA 15260 
(412) 648-7211 

Price 

There is a $5.00 copying charge. 

Source 

Lazar, M.K. and Bean, R.M. (1991). Alternative Assessment Measures in Adult Basic Educa- 
tional Programs. University of Pittsburgh: Institute for Practice and Research in Education. 
Meryl Lazar, Ph.D. Phone interview: 11/11/96 
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BRIGANCE 
Life Skills Inventory 



Author & Date 
Albert H. Brigance (1994). 

Purpose 

“To evaluate reading, writing, speaking, listening, comprehending, and computing skills in 
the context of everyday scenarios and job-seeking situations.” For use with vocational, 
secondary special Vacation, ESL, and adult education programs. 

Description 

Series of assessments designed to measure specific skills attainment. Areas covered include: 
reading grade placement, career awareness and self-understanding, job-seeking skills and 
knowledge, ratings scales (self-concept, attitudes, responsibility, job interview), reading 
skills, speaking and listening, pre-employment writing, and math skills and concepts. 

Administration 

Most assessments are meant to be individually administered in a one-on-one situation, but 
some are modifiable for groups. 

Alternate Forms 

A “Quick Screen” provides an overview of selected competencies. 

Reliability 

Information not available from publisher. 

Validity 

According to Brigance, “(this assessment) correlate(s) with CASAS woriq)lace literacy 
competency standards, SCANS Foundation Skills, and meet(s) the requirements of the 
Perkins Act.” 

Scoring & Interpretation 
Criterion-referenced. On-site scoring. 

Comments 

Goals and objectives software, which facilitates writing Individual Education Plans, is avail- 
able. 

Availability 

Curriculum Associates, Inc. 

P.O. Box 2(X)1 

North Billerica, MA 01862-0901 
(800) 225-0248 

Price 

Life Skills Inventory, $89.95. Learner Record Books (10 pack) $24.95. Supplemental 
materials and other discounts available. 

Source 

Curriculum Associates descriptive and promotional material. 
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BRIGANCE 

Employability Skills Inventory 



Author & Date 
Albert H. Brigance (1995). 

Purpose 

To measure work-related and job-seeking skills. 

Description 

This assessment measures 124 separate workplace skills, including: reading grade place- 
ment, career awareness and self-understanding, job- seeking skills and knowledge, reading 
skills, speaking and listening, pre-employment writing, and math skills and concepts. Ques- 
tions are posed in a work-related context. 

Administration 

Most assessments are meant to be individually administered in a one-on-one situation, but 
some are modifiable for groups. 

Alternate Forms 

A “Quick Screen” provides an overview of selected competencies. 



Reliability 

Information not available from publisher. 

Validity 

Information not available from publisher, however, the publisher states that this test corre- 
lates with the SCANS Foundation Skills, P.L.94-142, the JTPA, the Perkins Act, the Ameri- 
cans with Disabilities Act, the Fair Employment Law, and the Immigration Act. 

Scoring & Interpretation 
Criterion referenced. On-site scoring. 

Availability 

Curriculum Associates, Inc. 

P.O. Box 2001 

North Billerica, MA 01862-0901 
(800) 225-0248 

Price 

$89.95. Learner Record Books (10 pack) $24.95. Supplemental materials available. 
Source 

Curriculum Associates descriptive and promotional material. 
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ESLOA 

ESL Oral Assessment 



Author & Date 

Literacy Volunteers of America (LVA), 1995 (revised). 

Purpose 

To assess a non-native learner’s ability to speak and understand English. 

Description 

Measures both receptive and expressive language ability. Structured on fotu’ levels which 
focus on beginning and intermediate English. Written to reflect real-life situations. Useful 
for diagnosis, program evaluation, measurement of student progress. 

Administration 

Individual. Learner responds orally to questions. Test is terminated when the learner misses 
more than a specified number of questions. No time limit: approximately 20 minutes 
needed. 

Alternate Forms 
None. 

Reliability 
Not available. 

Validity 
Not available. 

Scoring & Interpretation 
Simple, quick scoring. 

Comments 

ESLOA literature states that the test was formulated using the “Ctommunicative Approach,” 
which emphasizes “natural or authentic interaction” using the English language. This ap- 
proach evaluates the learner’s ability to “negotiate meaning” in English. 

Availability 

Literacy Volunteers of America 
5795 Widewaters Parkway 
Syracuse, NY 13214 
(315) 445-8000 

Price 

(Hue books are $13.50 each; one answer sheet is provided with each book. Pads of 50 answer 
sheets are $3.50 each. 

Source 

LVA descriptive and promotional material. 
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Literacy Assessment Survey (LAS) 



Author & Date 

Kent Coperman, M. A. and Karen A. Ribarchik (1987). 

Purpose 

To measure the strengths and weaknesses of the adult literacy student reading below the third 
grade level. 

Description 

This test consists of two parts: an Arithmetic Whole Number Survey, and a Basic Reading 
Skills survey. The Arithmetic Whole Number Survey measures achievement in addition, 
subtraction, multiplication and division. The Basic Reading Skills Survey measures discrimi- 
nation skills, encoding, decoding, sequencing, phonics, and sentence comprehension. Both 
oral and silent reading skills are tested. 

Administration 

For individual administration only. Untimed. Consumable format. 

Alternate Forms 

Two forms. Form A and Form B, are available. 

Reliability 
None available. 

Validity 
None available. 

Scoring & Interpretation 

Manually scored by the test administrator. 

Comments 

This test can be used for placement into Adult Literacy, ABE, ESL and other programs. 

Availability 
Hadley Press 
Box 147 

Hadley, MI 48440 
(810) 232-6005 

Price 

$14.95 for 20 consumable booklets and one examiner’s manual. 

Source 

Hadley Press promotional materials. 



Tests of Adult Basic Education (TABE) 

Forms 7 and 8 



Author & Date 

McGraw Hill School Publishing Company (1994). 

Purpose , 

‘To measure achievement of basic skills commonly found in adult basic education cumcula 
and taught in high school and adult instructional programs.” 

Description 

Standardized, norm-referenced test which measures achievement in reading, language, 
spelling and mathematics computation and application. The TABE Complete Battery was 
revised in 1994 from the TABE Forms 5 and 6. Significant revisions: more life-skills-bas^ 
content; the addition of more charts, graphs, maps and forms; the combination of the Reading 
Vocabulary and Reading Comprehension tests; and the addition of a lower measurement 
level, the Literacy Level (L), which measures reading level up to the grade equivalent of 1 .9. 
In addition to Level L, the following levels are available: Level E (1.6 to 3.9 grade equiva- 
lents), Level M (3.6 to 6.9), Level D (6.6 to 8.9), & Level A (8.6 to 12.9). 

Administration 

This timed test takes three hours to complete. Students use paper and pencil, and fill out 
answer sheets. 

Alternate Forms 

Two versions of each form are available. In addition, the company offers TABE Survey 
Form, which is a shorter overview measurement of skills, TABE-PC for use with the com- 
puter, TABE Espanol, and a Large-Print Edition. 

Reliability 

KR-20 coefficients for internal reliability range from .75 to .96 across all subjects. 

Validity 

Both construct and content validity analyses were performed; test was compared with the 
California Achievement Test, 5th Edition. 

Scoring & Interpretation 

The TABE 7 and 8 was normed on four groups of adult learners: ABE participants, voca- 
tional/technical school attendees, adult and juvenile offenders, and college students. Be- 
tween 7,(XX) and 9,000 individuals took each of the subtests. Norm-referenced scores (scale 
scores, grade equivalents, percentiles, and stanines) are available. 

Availability 

Publishers Test Service at CTB/McGraw Hill 
2500 Garden Road 
Monterey, CA 93940 
(800) 538-9547 



Complete Battery test books are $54.50 per 25 for each level. Answer sheets are $19.(X) per 
25 for each level. Volume discounts are available. Supplementary materials can also be 
piuehased. 

Source 

TABE/McGraw Hill descriptive and promotional information. 
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Test of Applied Literacy Skill 
(TALS) 



Author & Date 

Educational Testing Service (1991). 

Purpose 

To assess reading skills in the areas of document, prose, and quantitative literacy. 
Description 

Related to the 1991 National Adult Literacy Survey, the TALS is adult in content and tone 
and uses open-ended questions to test applied reading and math abilities. 

Administration 

Timed. Can be used with either individuals or groups. Each subtest ^es 40 minutes to 
complete. There are a total of 67 questions in dl three subtests combined. 

Alternate Forms 

Form A and Form B available. 

Reliability 

Information not available from publisher. 

Validity 

Correlated to the National Adult Literacy Survey (NALS), the National Assessment of 
Educational Progress (NAEP) and the U.S. Department of Labor’s Workplace Literacy 
Assessment. Correlation co-efficients not available. 

Scoring & Interpretation 

Using a score which denotes the number of questions answered correctly, the administrator 
can convert the score into an indication of proficiency level. An interpretive profile, which 
relates the score to other skills that the learner possesses, can also be formulated. 

Comments 

Designed for individuals familiar with American cultural norms and concepts, the language 
complexity and situational contexts used may make the TALS difficult to use with ESL 
students. 

Availability 

Simon & Schuster Workplace Resources 
5 Columbia Circle 
New York, NY 10023-7780 
(800) 395-7042 

Price 

(1992 information was the most current available.) 10-packs of Test Batteries for Prose, 
Document, or Quantitative Literacy, $17.50 each. Complete tests sets (25 test batteries) 

$1 18.95 each. Administration and scoring manual, $15.00. 

Source 

Simon & Schuster promotional materials. 
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Wonderlic Basic Skills Test (WBST) 



Author & Date 

EUot R. Long, Victor S. Artese, Winifred L. Clonts, and Michael Callans (1994). 

Purpose 

To test the job-related basic skills of employment applicants. 

Description 

Test is “constructed to bridge the gap between school and work by measuring basic math and 
language skills for the wor^lace.” It “includes specific test content to meet identified job 
requirements,” and “measures how well skills are applied in the workplace.” The WB5T has 
been review^ for gender and ethnic bias. 

Administration 

Paper & pencil with two subtests: the Test of Verbal Skills, and the Test of Quantitative 
Stalls. Each takes 20 minutes to complete, and can be administered together or separately. 

Alternate Forms 

The Verbal Skills and the Quantitative Skills tests each have two forms. A “Skills Compos- 
ite” test, a shorter, combing version of the two tests, is also available. 

Reliability 

Test/retest correlations were conducted and studied to verify reliability. The average reliabil- 
ity coefficient for the Verbal Skills tests was .83; for the (Quantitative Skills test, it was .87. 

Validity 

The WBST is both criterion- and norm-referenced. The test was normed on students in 
school and adults at work; norms are presented by reference group and by job/training pro- 
gram title. Factor analyses, item functioning, item selection and scaling tests were performed 
to validate the WBST. 

Scoring & Interpretation 

Can be scored on-site, using a Scoring Diskette (IBM compatible) or sent to Wonderlic for 
scoring and interpretation. A validation service, evaluating the relationship between the job 
applicants test scores and their potential job performance, is available. Test results are tied to 
“required math and language skills listed in the U.S. Department of Labor’s Dictionary of 
Occupational Titles." The test can be scored by job requirements or grade level. The authors 
note that the WBST does not identify specific areas of skill weakness or assign specific levels 
of language or math training.” 

Availability 

Wonderlic Personnel Test, Inc. 

1509 N. Milwaukee Ave. 

Libertyville, EL 60048-1380 
(800)323-3742 Fax: (708) 680-9492 

Price 

Verbal Skills or Quantitative Skills tests purchased individually: $80.00 per 25. 

Skills Composite : $1 10.00 per 25. Prices include test booklets, answer sheets, manual, and 
scoring diskette. Discounts available for volume purchases. 

Source 

Wonderlic 1996 promotional material and WBST manual. 
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WorkKeys 



Author & Date 

American College Testing Program (ACT) (1994). 

Purpose 

To assess learners’ workplace skills. 

Description 

WorkKeys measures learners’ competencies through performance-based assessments in eight 
skills areas: Reading for Information, Applied Mathematics, Listening, Writing, Teamwork, 
Locating Information, Applied Technolo^, and Observation. 

Administration 

Each assessment takes between 35 to 45 minutes to complete. Sorne tests are paper-and- 
pencil only; others require listening to an audiotape or viewing a videotape to answer ques- 
tions. Questions on all assessments are multiple choice. 

Alternate Forms 
None. 

Reliability 

According to WorkKeys literature, “generalizability studies have found that G coefficients 
[appropriate for norm-referenced decisions] for Listening and Writing are .66 and .82, re- 
spectively. PHI coefficients [appropriate for criterion referenced decisions] are .62 for 
Listening and .81 for Writing.’ 

Validity 

Criterion-related validity studies “found statistically and practically significant relationships 
between assessment scores and job performance for [selected] jobs.” WorkKeys is also 
positively correlated to Ae ACT Assessment. 

Scoring & Interpretation 

Assessments must be sent into ACT for scoring. Computer scoring provides information on 
the learner’s sldll levels, descriptions of skills typically associated with those levels, and 
ideas/advice for instmction. 

Comments 

WorkKeys assessments include “skills identified by ... SCANS and by ACT panels.” 
Availability 

ACT — WorkKeys Client Services 
2201 N. Dodge St. 

P.O.Box 168 

Iowa City, lA 52243-0168 
(800)WORKKEY Fax: (319) 337-1725 

Price 

$4.00 per assessment (Reading, Applied Mathematics, Applied Technology, Locating 
Information); $7.00 per assessment (Teamwork, Observation); $13.50 per assessrnent (Lis- 
tening), $12.50 per assessment (Writing.) Complete set: $65.00. Video and audio tapes are 
extra. Assessment scoring is included in the price. 

Source 

Work Keys descriptive and promotional information. 
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Resources 



Resources 



Assessment 

ERIC Clearinghouse on Assessment and Evaluatjon 

Catholic University of America 

210 O’Boyle Hall 

Washington, DC 20064 

800-464-3742 Fax: 202-319-6692 

E-Mail: eric_ae@cua.edu Internet: http://www.cua.edu/www/eric_ae 

Provides research information, bibliographies, information and analysis papers, and other 
products and services on tests and other measurement devices; methodology or measurement 
and evaluation; application of tests, measurement, or evaluation in educational projects or 
programs; research design and methodology in the area of assessment and evaluation; and 
learning theory. 

ERIC Test Collection Clearinghouse 
Educational Testing Service 
Princeton, NJ 08541 
609-734-5737 Fax: 609-683-7186 
E-Mail: mhalpem@rosedale.org 

Prepares descriptions of commercially available and noncommercially available tests, 
checklists, instruments, questionnaires, and other assessment and evaluation tools. 



Adult Basic and Literacy Education 

ERIC Clearinghouse on Adult. Career, and Vocational Education 

The Ohio State University 

1900 Kenny Road 

Columbus, OH 43210-1090 

800-848-4815 Fax: 614-292-1260 

E-Mail: ericacve@magnus.acs.ohio-state.edu 

Provides information and resources for all levels and settings of adult and continuing, 
career, and vocational/technical education, including adult education (firom basic literacy 
training through professional skills upgrading), career awareness, career decision making and 
development, vocational and technical education, employment and training programs, youth 
employment, work experience programs, education/business partnerships, adult retraining 
and vocational rehabilitation for individuals with disabilities. 



ERIC Clearinghouse on Re ading. English, and Communication 

Indiana University 

Smith Research Center, Suite 150 

2805 East 10th Street 

Bloomington, IN 47408-2698 

800-759-4723 Fax: 812-855-4220 

E-Mail: ericcs@ucs.indiana.edu Internet: http://www.indiana.edu/~eric_rec 

Instructional materials, cimicula, tests/measurement, and methodology for all levels of 
reading. Emphasis on research and instructional development in reading, writing, speaking, 
and listening; identification, diagnosis, and remediation of reading problems; all aspects of 
reading behavior with emphasis on physiology, psychology, sociology, and teaching. 



English as a Second Language 

ERIC Cleari nghouse for ESL Literacy Education 

Center for Applied Linguistics 

1118 22nd Street NW 

Washington, DC 20037 

202-429-9292 X 200 Fax: 202-659-5641 

Internet: ncle@cal.org 

Addresses all aspects of literacy education for adults and out-of-school youth with 
limited English proficiency. Specific topics include development of reading, writing, com- 
putational, and communication skills; programs and projects in employment or vocational 
training; intergenerational literacy; and program design development, implementation, and 
evaluation; student assessment and placement; teaching methods and approaches including 
technology; ciuricula; learning styles; cross-cultural considerations, and research on second 
language literacy. 



Special Needs 

National Adult Literacy and Learning Disabilities Center (ALLD) 

Academy for Educational Development 
1875 Connecticut Avenue N.W. 

Washington, D.C. 20009-1202 
800-953-2553 or (202) 884-8185 
E-Mail: info@nalldc.aed.org 

The National ALLD Center, funded by the National Institute for Literacy, promotes 
awareness about the relationship between adult literacy and learning disabilities. Through 
its national information exchange network and technical assistance training, the Center helps 
literacy practitioners, policymakers, and researchers better meet the needs of adult with 
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learning disabilities. Resources include Screening for Adults with Learning Disabilities', 
Adults with Learning Disabilities: Definitions and Issues; and Techniques for Working with 
Adults with Learning Disabilities. 



GED Hotline 

American Council on Education 
One Dupont Circle 
Washington, DC 20036 
800-626-9433 Fax: 202-775-8578 

Provides 24 hour operator service with information on local GED classes and testing 
services. Tests are available in audio, Braille, and large print editions. They have an accom- 
modations guide for people taking the GED who have a learning disability. 

Job Accommodation Network 
918 Chesmut Ridge Road 
Suite 1 Box 6080 
Morgantown, WV 26505 
800-526-7234 

Provides a free consulting service with information on equipment, methods and modifi- 
cations for disables persons to improve their learning environment. Call from 8 AM to 8 PM 
(Monday through Thursday) and 8 AM to 5 PM (Friday) (ET). 

Orton Dyslexia Society 
8600 LaSalle Road 
Chester Building, Suite 382 
Baltimc»%, MD 2128602044 
800-222-3123 

Twenty-four hour voice mail service that receives information requests. From 8:30 AM 
to 4:30 PM Monday-Friday (ET), call (410) 296-0232 to receive information about appro- 
priate materials concerning dyslexia. Provides information on publications, referrals for 
testing and tutors, local branches of ODS, and workshops and conferences. 
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Disability Deflnitions and Checklists 



Disability DeHnitions and Checklists 



The following definitions were adapted from Salvia and Ysseldyke (1995), the Individu- 
als with Disabilities in Education Act — IDEA (34 CFR §300.7), and information from the 
National Adult Literacy and Learning Disabilities Center. 

Attention Deficit Disorder (ADD) 

Students with ADD have a disability of the central nervous system which affects their 
behavior and educational performance, most noticeably through inattention and an inability 
to complete tasks. Other characteristics include a short attention span, inattention to details, 
difficulty following directions, forgetfulness, and trouble with organization. 

Attention Deficit Hyperactivity Disorder (ADHD) 

Students with ADHD display many of the same characteristics as those with ADD as 
well as developmentally inappropriate inattention, impulsivity, and overactivity that affects 
educational performance. 



Autism 

Autistic students often perform repetitive movements, are extremely resistant to environ- 
mental change or change in daily routine, and have unusual responses to sensory experi- 
ences. They are usually evaluated by language specialists and psychologists. The term 
autism doesn’t apply if the student’s educational performance is adversely affected by a 
serious emotional dismrbance. Autism is difficult to distinguish from severe forms of 
mental retardation if the student has limited intellectual ability. 

Deafness and Hearing Impairment 

Deafness is an impairment in hearing so severe that the individual is impaired in process- 
ing linguistic information through hearing, with or without amplification. A student with a 
hearing impairment has trouble hearing, whether permanent or fluctuating, that adversely 
affects her/his educational performance. 

Emotional Disturbance 

EnK)tionally disturbed individuals display one or more of the following characteristics 
over a long period of time and to a degree that adversely affects educational performance: 
an inability to learn which cannot be explained by intellectual, sensory, or health facts; an 
inability to build or maintain satisfactory interpersonal relationships with peers and teachers; 
inappropriate types of behavior or feelings under normal circumstances; a general pervasive 
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mood of unhappiness or depression; or, a tendency to develop physical symptoms or fears 
associated with personal or school problems. 

Learning Disability 

Learning disabled students have significant difficulties acquiring or using listening, 
speaking, reading, writing, reasoning, mathematical, or social skills. Learning disabilities are 
presumed to be due to a central nervous system dysfunction. They may occur with other 
handicapping conditions (e.g., sensory impairment, mental retardation, social and emotional 
disturbances), with socio-environmental influences (e.g., cultural differences, insufficient or 
inappropriate instruction), and especially with attention deficit disorders; however, the 
learning disability is not the direct result of these conditions or influences. 

Mental Retardation 

Mentally retarded students demonstrate significantly subaverage intellectual functioning 
along with deficits in adaptive behavior. Students who are eventually labeled mentally 
retarded are often referred because of generalized slowness in academic achievement, social 
and emotional and perhaps physical development, and language ability. An assessment for 
mental retardation must assess achievement, intelligence, and adaptive behavior. 

Speech or Language Impairment 

An individual with a speech or language impairment has a communication disorder such 
as stuttering, lisping, voice impairment (voice is too high or low, too soft or loud), or prob- 
lems articulating speech or language. 

Traumatic Brain Injury 

Individuals with traumatic brain injury have an injury to the brain caused by external 
physical force which resulted in total or partial disability or psychological impairment, or 
both. The term applies to open or closed head injuries resulting in impairments in one or 
more areas cognition, language, memory, attention, reasoning, abstract thinking, judgment, 
problem solving, sensory or perceptual abilities, psychosocial behavior, physical functions, 
information processing, or speech. The term does not apply to brain injuries that are con- 
genital or induced by birth trauma. 

Visual Impairment 

An individual with a visual impairment has a problem with vision that, even with correc- 
tion, hinders educational performance. It includes both partial sight and blindness. 



O 




109 



Disability Checklists 



These checklists were adapted from the National Adult Literacy and Learning Disabilities 
Center 1995 publication, Screening for Adults with Learning Disabilities: The Role of the 
Practitioner in the Assessment Process. The checklists should be used for screening pur- 
poses only. The behaviors should be observed over time; several behaviors should be 
observed before suspecting a disability. It may be helpful to make written notes in addition 
to using the checklists. 



Indicators of Visual Impairment 

Q squinting 

Q eyes water or become red after a short period of work 
complains of blurred vision 
complains of tired or sore eyes: rubs eyes a lot 
Q holds reading material either very close or very far 
closes one eye while reading 
letters are written in uncustomary manner 

Indicators of Hearing Impairment 

asks for repetition of conversation 
turns one ear toward the speaker 
difficulty in spelling 
misunderstands speech 
speaks loudly 

Q oral reading is choppy; words skipped, endings left off, repetitions 

Indicators of Other Physical Impairments 

Q orthopedic impairments 
Q chronic illness 

seizures, fainting, or generalized weakness 
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Indicators of Attention Deficit Disorder (ADD) 

difficulty paying attention 
Q short attention span 
easily distracted 
Q forgetful 

inattentive to details 
makes careless mistakes 
Q difficulty listening 

difficulty following directions 
difficulty with organization 
Q misplaces things 
doesn’t finish tasks 



Indicators of Attention Deficit Hyperactivity Disorder (ADHD) 

(Same as ADD plus:) 
hyperactivity 

excessive movement, fidgets, squirms 
Q excessive talking 

speaks and acts out of turn, interrupts 
difficulty with quiet activities 
Q difficulty remaining seated 
Q active at inappropriate times 
overly impulsive 

engages in physically dangerous activities; doesn’t consider consequences 



Indicators of Learning Disabilities 



Academic 

reading problems: loses place; reads words or syllables backwards; substitutes, deletes, 
or transposes letters and syllables; rereads or reads very slowly; reading style is halting 
or jerky; guesses words frequently with compromised comprehension 
expressive language problems: writes letters or numbers backwards; spells phonetically 
and cannot remember spelling patterns; reverses letters in spelling; punctuation errors; 
poor handwriting; poor writing organization 

Q math problems: trouble remembering facts and procedures; difficulty copying numbers 
and working with numbers in columns; trouble with lefl/right orientation; confuses 
similar numbers or symbols; reads numbers backwards; trouble following sequential 
procedures and directions with multiple steps 

Physical 

Q visual discrimination: inability to detect differences in forms, letters, and words 
Q visual memory: trouble retaining a full mental image of what was seen 
Q auditory discrimination: inability to recognize the differences between sounds 
Q auditory memory: inability to store and recall what was heard 

Behavioral 
Q low self-esteem 

Q difficulty concentrating or easily distracted 
Q unpredictable performance 

Q poor motivation or extreme drive to complete a task 
Q displays memory problems 
Q difficulty sitting still 

Q directionality problems (left/right, up/down/, north/south/east/west) 

Social 

misinterprets what others say (tone of voice, facial expressions) 

Q lacks awareness of personal space 
Q difficulty in establishing friendships 
Q excessive mood swings 

inappropriate reactions to the moods of others 
Q trouble understanding humor 
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